4 Configuring a Clustered Deployment

This section describes the tasks that you must perform to configure Oracle Service Bus for deployment in a clustered environment.

Note:

If you want to run Oracle Service Bus on a managed server in a non-clustered environment, see Chapter 2, "Configuring a Non-Clustered Deployment."

After planning the architecture of your clustered domain, as described in Section 3.2, "Designing a Clustered Deployment," you are ready to set up Oracle Service Bus in a clustered environment. To do this, you must configure an administration server and managed servers, and then deploy Oracle Service Bus resources to the servers. You also need a router (hardware or software), if you need inbound HTTP load balance functions. The persistent configuration for a domain of Oracle WebLogic Server instances and clusters is stored in an XML configuration file (config.xml) in the config directory of the root directory of your Oracle Service Bus domain.

To set up and deploy Oracle Service Bus in a clustered domain, complete the following steps:

Section 4.1, "Step 1. Comply with Configuration Prerequisites"
Section 4.2, "Step 2. Prepare an Oracle Service Bus Domain"
Section 4.3, "Step 3. Configure Oracle Service Bus Security"
Section 4.4, "Step 4. Starting, Stopping, and Monitoring Managed Servers"
Section 4.5, "Step 5. Deploy an Oracle Service Bus Configuration"
Section 4.6, "Step 6. Update Your Domain as Your Production Environment Changes"

For information about deploying Oracle Service Bus in a non-clustered environment, see Chapter 2, "Configuring a Non-Clustered Deployment."

4.1 Step 1. Comply with Configuration Prerequisites

This section describes prerequisites for configuring Oracle Service Bus to run in a clustered environment:

Obtain an IP address for the administration server you will use for the cluster.

All Oracle WebLogic Server instances in a cluster use the same administration server for configuring and monitoring. When you add servers to a cluster, you must specify the administration server that each will use.
Define a multicast address for each cluster.

Note:
You are prompted to provide a multicast address when you create an Oracle Service Bus domain using the Oracle Fusion Middleware Configuration Wizard. (See Section 4.2, "Step 2. Prepare an Oracle Service Bus Domain.")
The multicast address is used by cluster members to communicate with each other. Clustered servers must share a single, exclusive, multicast address. For each cluster on a network, the combination of multicast address and port must be unique. If two clusters on a network use the same multicast address, they should use different ports. If the clusters use different multicast addresses, they can use the same port or accept the default port, 7001. To support multicast messages, the administration server and the managed servers in a cluster must be located on the same subnet.
Define IP addresses for the servers in your cluster. You can do this in a number of ways:

Note:
You are prompted to provide listen addresses for servers when you create a Oracle Service Bus domain using the Oracle Fusion Middleware Configuration Wizard. (See Section 4.2, "Step 2. Prepare an Oracle Service Bus Domain.")
- Assign a single IP address and different listen port numbers to the servers in the cluster.
  
  By assigning a single IP address for your clustered servers with a different port number for each server, you can set up a clustered environment on a single machine without the need to make your machine a multi-homed server.
  
  To access such an IP address from a client, structure the IP address and port number in your URL in one of the following ways:
  - ipAddress:portNumber-portNumber – When the port numbers are sequential. For example:
    
    127.0.0.1:7003-7005
  - ipAddress:portNumber+...+portNumber – When the port numbers are not sequential. For example:
    
    127.0.0.1:7003+7006+7008
  - ipAddress:portNumber,ipAddress:portNumber,... – Verbose, explicit specification. For example:
    
    127.0.0.1:7003,127.0.0.1:7004,127.0.0.1:7005
- Assign a static IP address for each Oracle WebLogic Server instance to be started on each machine in the cluster.
  
  In this case, when multiple servers are run on a single machine, that machine must be configured as a multi-homed server, that is, multiple IP addresses are assigned to a single computer. Under these circumstances, structure the cluster address as a comma-separated list of IP addresses.
  
  For example, the following listing is an example of a cluster address specified in a config.xml file. It specifies a static IP address for each of the four servers in a cluster named MyCluster:
```
<Cluster ClusterAddress="127.0.0.1:7001,127.0.0.2:7001,127.0.0.3,127.0.0.4:7001" Name="MyCluster"/>
```
  You can also use a DNS approach to identifying servers.
  
  For more information on addressing issues, see "Avoiding Listen Address Problems" in "Setting Up WebLogic Clusters" in Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.
  
  Note:
  In test environments, it is possible to have multiple Oracle WebLogic Server instances on a single machine. In these circumstances, you can have some Oracle WebLogic Server instances on the same node with different port numbers and some on different nodes with the same port number.
Configure one of the following databases for your clustered domain:
- Microsoft SQL Server
- Oracle
  
  Note:
  The local copy of the Apache Derby database that is installed with Oracle WebLogic Server is for evaluation purposes only.
It is important to configure your database appropriately for production use. You must provide adequate space to store data and log messages, and follow best practices for administering your database.

Note:
You can configure your database to use concurrent access.

For the latest information about issues regarding specific databases, see the product release notes.
Include a shared file system. A shared file system is required for any cluster you want to be highly available. We recommend either a Storage Area Network (SAN) or a multiported disk system.

For information about configuring a highly available cluster, see "Configuring WebLogic JMS Clustering" in "Configuring Advanced JMS System Resources" in Oracle Fusion Middleware Configuring and Managing JMS for Oracle WebLogic Server.
Configure a hardware or software router for your system. Load balancing can be accomplished using either the built-in load balancing capabilities of a WebLogic proxy plug-in or separate load balancing hardware.

For information about hardware and software routers, see Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

Note:
Additional requirements apply when you design your domain to include one or more firewalls. For a description of how to add firewall information to your domain configuration file, see Section 4.2.1.1, "Adding Proxy Server or Firewall Information to your Domain Configuration." For additional information, see "Communications in a Cluster" in Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.
Oracle Service Bus load balances File, Email, and FTP transport processing across the managed servers in a cluster. All managed servers in the cluster should be able to access the Archive, Stage, and Error directories specified in any File, Email, or FTP proxy service configuration. These directories should be configured in a shared file system such as NFS. By using a shared file system, users and programs can access files on remote systems almost as if they were local files.

For more information about setting up clustered Oracle WebLogic Server instances, see "Setting Up WebLogic Clusters" in Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

4.2 Step 2. Prepare an Oracle Service Bus Domain

When preparing an Oracle Service Bus domain, you must add a definition for each managed server to the domain configuration file (config.xml), assign all managed servers to a cluster, specify the Oracle Service Bus components on the servers in your domain, and so on.

To prepare an Oracle Service Bus environment in a clustered domain, complete the tasks described in the following sections:

Section 4.2.1, "Creating an Oracle Service Bus Domain Using the Oracle Fusion Middleware Configuration Wizard"
Section 4.2.2, "Configuring JMS Resources"

4.2.1 Creating an Oracle Service Bus Domain Using the Oracle Fusion Middleware Configuration Wizard

For information on creating an Oracle Service Bus domain with the Oracle Fusion Middleware Configuration Wizard, see "Installing and Configuring Oracle Service Bus 11g" in the Oracle Fusion Middleware Installation Guide for Oracle Service Bus.

4.2.1.1 Adding Proxy Server or Firewall Information to your Domain Configuration

If you will be using Web services behind a proxy server or firewall, you must edit the config.xml file to include information about that proxy server or firewall.

To add proxy server or firewall information to your domain configuration, complete the following steps:

Open config.xml with an ASCII editor.
Find the line that starts with the following tag in the config.xml file:
```
<Cluster
```

Add the following three attributes to the Cluster attribute list:

FrontendHTTPPort="proxyPort" FrontendHTTPSPort="proxySSLPort" FrontendHost="proxyServerHost"

For example, the following listing is an example of a cluster address with a firewall specified in a config.xml file for a cluster named MyCluster and a proxy server named MyProxy:

<Cluster ClusterAddress="127.0.0.1:7001,127.0.0.2:7001,127.0.0.3,127.0.0.4:7001" FrontendHTTPPort="7006" FrontendHTTPSPort="7007" FrontendHost="MyProxy" MulticastAddress="127.0.0.5" MulticastPort="7010" Name="MyCluster"/>

Save your changes and close the config.xml file.

4.2.2 Configuring JMS Resources

In addition to configuring JMS file stores in the Oracle Fusion Middleware Configuration Wizard, proxy services and business services that use JMS require configuration of the following resources:

JMS queues/topics. Oracle Service Bus automatically configures JMS queues for proxy services that are implemented using JMS. You must configure JMS queues/topics for all business services using JMS and for any proxy services that are implemented using non- JMS.

Proxy services can consume messages from a remote queue on a separate domain. In this case, Oracle Service Bus will not create the queue for you. The JMS queues can be created for proxy services only if the queues are on the same local Oracle Service Bus domain.
JMS connection factories. You must configure JMS connection factories for all business services and proxy services implemented using JMS.

For information about configuring JMS resources, see "Configuring Advanced JMS System Resources" in Oracle Fusion Middleware Configuring and Managing JMS for Oracle WebLogic Server.

4.3 Step 3. Configure Oracle Service Bus Security

Oracle Service Bus leverages the security features of Oracle WebLogic Server to ensure message confidentiality and integrity (message-level security), secure connections between clients and Oracle WebLogic Server (transport-level security), and authentication and authorization (access control). For information about the tasks you must complete, see "Security" in the Oracle Fusion Middleware Developer's Guide for Oracle Service Bus.

Note:

You must configure security separately for each Oracle Service Bus domain. Oracle Service Bus does not export or import security configurations.

If you want to configure SSL for your cluster, you can do so when creating your domain or by using the Oracle WebLogic Server Administration Console. For a domain in which security functionality is deployed in a multi-node cluster, you also need to configure keystores, server certificate and private key for each managed server, and so on, for every machine in a cluster. You either need to use a separate keystore for each machine or you can use a single keystore if it is available to all machines.

4.4 Step 4. Starting, Stopping, and Monitoring Managed Servers

This section describes the basic management tasks for the managed servers in your clustered domain:

Section 4.4.1, "Starting and Stopping Managed Servers"
Section 4.4.2, "Monitoring Your Servers"

4.4.1 Starting and Stopping Managed Servers

Node Manager is a utility that enables you to start, stop, and migrate your Oracle WebLogic Server instances. You can start your managed servers using Node Manager in conjunction with the Oracle WebLogic Server Administration Console, or you can create WLST scripts to automate Node Manager functionality.

Tip:

Run the setDomainEnv script before the startNodemanager script to ensure that the Oracle Service Bus classes are available to spawned servers. Alternatively, explicitly set classpath in the Oracle WebLogic Server Administration Console before attempting to start the server.

By default, when the Oracle Fusion Middleware Configuration Wizard generates an Oracle Service Bus cluster domain:

The Domain Singleton Marker Application (domainsingletonmarker.ear) is targeted to the first managed server in the cluster. For data aggregation to function properly, the server that domainsingletonmarker.ear is targeted to must be started first and must be available when other managed servers are started.
The PurgingMDB (msgpurger.ear) is targeted to the first managed server in the cluster. For message purging to function properly, the server where msgpurger.ear is targeted must be available.

For more information on Node Manager, see "Using Node Manager to Control Servers" in Oracle Fusion Middleware Node Manager Administrator's Guide for Oracle WebLogic Server. For a complete overview of methods to start and stop managed servers, see "Starting and Stopping Servers" in Oracle Fusion Middleware Managing Server Startup and Shutdown for Oracle WebLogic Server.

4.4.2 Monitoring Your Servers

Once startup is complete, you can use the Oracle Service Bus Console to verify the status of servers. For information about using Oracle Service Bus Console to monitor your servers, see "Listing and Locating Servers" in "Monitoring" in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

4.5 Step 5. Deploy an Oracle Service Bus Configuration

You deploy an Oracle Service Bus configuration in a clustered environment following the same procedure as for a non-clustered deployment. For a description of the deployment procedure, see Section 2.4, "Step 4. Deploy an Oracle Service Bus Configuration."

Note:

If you have imported a configuration from a non-clustered environment and that configuration includes proxy services that use File, FTP, or Email transports, you must specify a Managed Server for each of those proxy services. The Managed Server list appears in the Oracle Service Bus Console in clustered Oracle Service Bus domains only.

4.6 Step 6. Update Your Domain as Your Production Environment Changes

Production environments change over time and as application use increases. This section describes how to update your domain in response to common production environment change scenarios:

Section 4.6.1, "Adding a Managed Server"
Section 4.6.2, "Deleting a Managed Server"
Section 4.6.3, "Changing a Business Service in a Cluster"
Section 4.6.4, "Installing a New Version of a Proxy Service in a Cluster"

4.6.1 Adding a Managed Server

As use of Oracle Service Bus grows, you can add new managed servers to your Oracle Service Bus cluster to increase capacity. For detailed instructions, see "Scaling the Topology" in the Oracle Fusion Middleware High Availability Guide.

After you scale your topology, use the instructions in the following topics to update business and proxy services.

4.6.1.1 Updating Business Service Configurations for an Expanded Cluster

If your Oracle Service Bus configuration includes one or more business services that use JMS request/response functionality, then you must also perform the following procedure using the Oracle Service Bus Console after adding the new managed server to the cluster:

In the Change Center, click Create to create a session.
Using the Project Explorer, locate and select a business service that uses JMS request/response. Business services of this type display Messaging Service as their Service Type.
At the bottom of the View Details page, click Edit.
If there is a cluster address in the endpoint URI, add the new server to the cluster address.
On the Edit a Business Service - Summary page, click Save.
Repeat the previous steps for each remaining business service that uses JMS request/response.
In the Change Center, click Activate.

The business services are now configured for operation in the extended domain.

Note:

For business services that use a JMS MesageID correlation scheme, you must edit the connection factory settings to add an entry to the table mapping managed servers to queues. For information on how to configure queues and topic destinations, see "JMS Server Targeting" in Oracle Fusion Middleware Configuring and Managing JMS for Oracle WebLogic Server.

4.6.1.2 Updating Proxy Service Configurations for an Expanded Cluster

If your Oracle Service Bus configuration includes one or more proxy services that use JMS endpoints with cluster addresses, then you must also perform the following procedure using the Oracle Service Bus Console after adding the new managed server to the cluster:

In the Change Center, click Create to create a session.
Using the Project Explorer, locate and select a proxy service that uses JMS endpoints with cluster addresses.
At the bottom of the View Details page, click Edit.
If there is a cluster address in the endpoint URI, add the new server to the cluster address.
On the Edit a Proxy Service - Summary page, click Save.
Repeat the previous steps for each remaining proxy service that uses JMS endpoints with cluster addresses.
In the Change Center, click Activate.

The proxy services are now configured for operation in the extended domain.

4.6.2 Deleting a Managed Server

Using Oracle WebLogic Server administration tools, you can delete a managed server from your Oracle Service Bus cluster. Before deciding to delete a managed server, you should take into account the following considerations:

If your Oracle Service Bus configuration includes one or more proxy services that use File, FTP, or Email transports that have pinned transport pollers to the managed server that you want to remove from the cluster, then you must select a different managed server for each of those proxy services before removing the managed server from the cluster.
The managed server that hosts the Domain Singleton Marker Application must not be deleted from the cluster. If the managed server that hosts the Domain Singleton Marker Application fails, you must perform a manual migration.
If you want to delete the managed server that hosts the Message Reporting Purger application, you must select a different manager server for the Message Reporting Purger and its associated queue (wli.reporting.purge.queue).

In the following procedure, you will delete resources associated with a managed server, and finally delete the managed server itself. The names of most affected resources end with _auto_x, where x is the number of the managed server you want to delete. For example, the JMS reporting provider queue for managed server 2 is called wli.reporting.jmsprovider.queue_auto_2.

Note:

The Oracle WebLogic Server Console by default displays only a partial list of resources. To display a full list of resources, click Customize this table and increase the Number of rows displayed per page.

To delete a managed server from a cluster:

After you have read the considerations earlier in this section, stop all managed servers in the cluster. Keep the Admin server running.
Log into the Oracle WebLogic Server Console, and click Lock & Edit in the Change Center.
Delete all *_auto_x queue members from distributed queues.
1. In the Domain Structure pane, select Services > Messaging > JMS Modules to display a list of JMS modules.
2. Click the jmsResources module. You will see a list of distributed queues whose names begin with dist_. For example, dist_QueueIn_auto.
3. Click a distributed queue and go to its Members tab. Delete the queue for the managed server. For example, if you are deleting managed server 2, delete QueueIn_auto_2 from the dist_QueueIn_auto distributed queue.
4. Repeat these steps for each distributed queue, deleting the relevant *_auto_x resource for the managed server you are deleting.
Delete all *_auto_x queues.
1. In the list of resources for the jmsResources module, delete all queues whose names end with *_auto_x for the managed server you are deleting. For example, delete wli.reporting.jmsprovider.queue_auto_2 if you are deleting managed server 2.
2. Delete all *_auto_x queues from the Services > Messaging > JMS Modules > WseeJmsModule for the managed server you are deleting. For example, delete DefaultCallbackQueue-WseeJmsServer_auto_2 and DefaultQueue-WseeJmsServer_auto_2 if you are deleting managed server 2.
Delete all JMS subdeployments for the managed server you are deleting.
1. Select Services > Messaging > JMS Modules.
2. Click the configwiz-jms module, and click the Subdeployments tab.
  
  In the Targets column, note the *_auto_x targets. Delete all subdeployments targeted to *_auto_x for the managed server you are deleting.
3. As with the previous step, delete all *_auto_x subdeployments from the jmsResources and WseeJmsModule modules for the managed server you are deleting.
Delete all JMS servers targeted to the managed server you are deleting.
1. Select Services > Messaging > JMS Servers.
2. Delete all *_auto_x JMS servers for the managed server you are deleting.
Delete all store-and-forward agents targeted to the managed server you are deleting.
1. Select Services > Messaging > Store-and-Forward Agents.
2. Delete all *_auto_x store-and-forward agents for the managed server you are deleting.
Delete all persistent stores targeted to the managed server you are deleting.
1. Select Services > Persistent Stores.
2. Delete all *_auto_x persistent stores for the managed server you are deleting.
Delete all migratable targets for the managed server you are deleting.
1. Select Environment > Migratable Targets.
2. Delete server_name (migratable) for the managed server you are deleting. For example, if the managed server you are deleting is named osb_server2, delete osb_server2 (migratable).
Untarget the managed server from the ALSB Framework Starter Application.
1. In the Domain Structure pane, click Deployments.
2. Click ALSB Framework Starter Application, and click the Targets tab.
3. Select the ALSB Framework Starter Application check box, and click Change Targets.
4. Remove the managed server as a deployment target.
  
  Note:
  If the All servers in the cluster targeting option is selected, you do not need to untarget the managed server.
Remove the managed server from the cluster.
1. Select Environment > Clusters, and click the name of the cluster.
2. Click the Servers tab, and delete the managed server from the cluster.
3. Go to the Configuration > General tab for the cluster.
  
  In the Cluster Address field, remove the managed server from the cluster address.
  
  In the Number Of Servers In Cluster Address field, reduce the number of servers by one.
Delete the managed server. For instructions, see "Delete Managed Servers" in the Oracle Fusion Middleware Oracle WebLogic Server Administration Console Online Help.
Restart the remaining managed servers.

4.6.3 Changing a Business Service in a Cluster

The procedure for changing a business service is the same in both non-clustered and cluster environments. For information about changing a business service, see Section 2.5.1, "Changing a Business Service."

However, the procedure for deploying changes to a business service in a cluster depends on the types of changes made to the business service and the nature of any other changes that might be deployed simultaneously. For more information, see the description of installation strategies in the following section.

4.6.4 Installing a New Version of a Proxy Service in a Cluster

As your business requirements change, you may need to make changes to your proxy services. You can make these changes dynamically online, partially offline, or completely offline. If your changes are backward compatible (that is, you are making no changes to interfaces), you can make your changes dynamically online using the Oracle Service Bus Console. Making other types of changes should be done partially or completely offline, which requires additional system administration steps.

For information about performing online updates, see Section 2.5.3, "Online Configuration Updates."

Making changes that include non-backward compatible changes to proxy service interfaces requires complete offline deployment. To install the new version, follow the procedure below while all servers are operational:

Quiesce all inbound messages.
Confirm all asynchronous backlogged messages have been processed.
Make the necessary changes in the proxy service, and test to verify the proxy service operates as required.
Resume accepting inbound messages.

For more information about backward compatibility and installation strategies, see Section 2.5.2, "Installing a New Version of a Proxy Service."