4 Configuring a Clustered Deployment

This section describes the tasks that you must perform to configure Oracle Service Bus for deployment in a clustered environment.

Note:

If you want to run Oracle Service Bus on a managed server in a non-clustered environment, see Chapter 2, "Configuring a Non-Clustered Deployment."

After planning the architecture of your clustered domain, as described in Section 3.2, "Designing a Clustered Deployment," you are ready to set up Oracle Service Bus in a clustered environment. To do this, you must configure an administration server and managed servers, and then deploy Oracle Service Bus resources to the servers. You also need a router (hardware or software), if you need inbound HTTP load balance functions. The persistent configuration for a domain of Oracle WebLogic Server instances and clusters is stored in an XML configuration file (config.xml) in the config directory of the root directory of your Oracle Service Bus domain.

To set up and deploy Oracle Service Bus in a clustered domain, complete the following steps:

Section 4.1, "Step 1. Comply with Configuration Prerequisites"
Section 4.2, "Step 2. Prepare an Oracle Service Bus Domain"
Section 4.3, "Step 3. Configure Oracle Service Bus Security"
Section 4.4, "Step 4. Starting, Stopping, and Monitoring Managed Servers"
Section 4.5, "Step 5. Deploy an Oracle Service Bus Configuration"
Section 4.6, "Step 6. Update Your Domain as Your Production Environment Changes"

For information about deploying Oracle Service Bus in a non-clustered environment, see Chapter 2, "Configuring a Non-Clustered Deployment."

4.1 Step 1. Comply with Configuration Prerequisites

This section describes prerequisites for configuring Oracle Service Bus to run in a clustered environment:

Obtain an IP address for the administration server you will use for the cluster.

All Oracle WebLogic Server instances in a cluster use the same administration server for configuring and monitoring. When you add servers to a cluster, you must specify the administration server that each will use.
Define a multicast address for each cluster.

Note:
You are prompted to provide a multicast address when you create an Oracle Service Bus domain using the Oracle Fusion Middleware Configuration Wizard. (See Section 4.2, "Step 2. Prepare an Oracle Service Bus Domain.")
The multicast address is used by cluster members to communicate with each other. Clustered servers must share a single, exclusive, multicast address. For each cluster on a network, the combination of multicast address and port must be unique. If two clusters on a network use the same multicast address, they should use different ports. If the clusters use different multicast addresses, they can use the same port or accept the default port, 7001. To support multicast messages, the administration server and the managed servers in a cluster must be located on the same subnet.
Define IP addresses for the servers in your cluster. You can do this in a number of ways:

Note:
You are prompted to provide listen addresses for servers when you create a Oracle Service Bus domain using the Oracle Fusion Middleware Configuration Wizard. (See Section 4.2, "Step 2. Prepare an Oracle Service Bus Domain.")
- Assign a single IP address and different listen port numbers to the servers in the cluster.
  
  By assigning a single IP address for your clustered servers with a different port number for each server, you can set up a clustered environment on a single machine without the need to make your machine a multi-homed server.
  
  To access such an IP address from a client, structure the IP address and port number in your URL in one of the following ways:
  - ipAddress:portNumber-portNumber – When the port numbers are sequential. For example:
    
    127.0.0.1:7003-7005
  - ipAddress:portNumber+...+portNumber – When the port numbers are not sequential. For example:
    
    127.0.0.1:7003+7006+7008
  - ipAddress:portNumber,ipAddress:portNumber,... – Verbose, explicit specification. For example:
    
    127.0.0.1:7003,127.0.0.1:7004,127.0.0.1:7005
- Assign a static IP address for each Oracle WebLogic Server instance to be started on each machine in the cluster.
  
  In this case, when multiple servers are run on a single machine, that machine must be configured as a multi-homed server, that is, multiple IP addresses are assigned to a single computer. Under these circumstances, structure the cluster address as a comma-separated list of IP addresses.
  
  For example, the following listing is an example of a cluster address specified in a config.xml file. It specifies a static IP address for each of the four servers in a cluster named MyCluster:
```
<Cluster ClusterAddress="127.0.0.1:7001,127.0.0.2:7001,127.0.0.3,127.0.0.4:7001" Name="MyCluster"/>
```
  You can also use a DNS approach to identifying servers.
  
  For more information on addressing issues, see "Avoiding Listen Address Problems" in "Setting Up WebLogic Clusters" in Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.
  
  Note:
  In test environments, it is possible to have multiple Oracle WebLogic Server instances on a single machine. In these circumstances, you can have some Oracle WebLogic Server instances on the same node with different port numbers and some on different nodes with the same port number.
Configure one of the following databases for your clustered domain:
- Microsoft SQL Server
- Oracle
  
  Note:
  The local copy of the Apache Derby database that is installed with Oracle WebLogic Server is for evaluation purposes only.
It is important to configure your database appropriately for production use. You must provide adequate space to store data and log messages, and follow best practices for administering your database.

Note:
You can configure your database to use concurrent access.

For the latest information about issues regarding specific databases, see the product release notes.
Include a shared file system. A shared file system is required for any cluster you want to be highly available. We recommend either a Storage Area Network (SAN) or a multiported disk system.

For information about configuring a highly available cluster, see "Configuring WebLogic JMS Clustering" in "Configuring Advanced JMS System Resources" in Oracle Fusion Middleware Configuring and Managing JMS for Oracle WebLogic Server.
Configure a hardware or software router for your system. Load balancing can be accomplished using either the built-in load balancing capabilities of a WebLogic proxy plug-in or separate load balancing hardware.

For information about hardware and software routers, see Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

Note:
Additional requirements apply when you design your domain to include one or more firewalls. For a description of how to add firewall information to your domain configuration file, see Section 4.2.1.1, "Adding Proxy Server or Firewall Information to your Domain Configuration." For additional information, see "Communications in a Cluster" in Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.
Oracle Service Bus load balances File, Email, and FTP transport processing across the managed servers in a cluster. All managed servers in the cluster should be able to access the Archive, Stage, and Error directories specified in any File, Email, or FTP proxy service configuration. These directories should be configured in a shared file system such as NFS. By using a shared file system, users and programs can access files on remote systems almost as if they were local files.

For more information about setting up clustered Oracle WebLogic Server instances, see "Setting Up WebLogic Clusters" in Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

4.2 Step 2. Prepare an Oracle Service Bus Domain

When preparing an Oracle Service Bus domain, you must add a definition for each managed server to the domain configuration file (config.xml), assign all managed servers to a cluster, specify the Oracle Service Bus components on the servers in your domain, and so on.

To prepare an Oracle Service Bus environment in a clustered domain, complete the tasks described in the following sections:

Section 4.2.1, "Creating an Oracle Service Bus Domain Using the Oracle Fusion Middleware Configuration Wizard"
Section 4.2.2, "Configuring JMS Resources"

4.2.1 Creating an Oracle Service Bus Domain Using the Oracle Fusion Middleware Configuration Wizard

For information on creating an Oracle Service Bus domain with the Oracle Fusion Middleware Configuration Wizard, see "Installing and Configuring Oracle Service Bus 11g" in the Oracle Fusion Middleware Installation Guide for Oracle Service Bus.

4.2.1.1 Adding Proxy Server or Firewall Information to your Domain Configuration

If you will be using Web services behind a proxy server or firewall, you must edit the config.xml file to include information about that proxy server or firewall.

To add proxy server or firewall information to your domain configuration, complete the following steps:

Open config.xml with an ASCII editor.
Find the line that starts with the following tag in the config.xml file:
```
<Cluster
```

Add the following three attributes to the Cluster attribute list:

FrontendHTTPPort="proxyPort" FrontendHTTPSPort="proxySSLPort" FrontendHost="proxyServerHost"

For example, the following listing is an example of a cluster address with a firewall specified in a config.xml file for a cluster named MyCluster and a proxy server named MyProxy:

<Cluster ClusterAddress="127.0.0.1:7001,127.0.0.2:7001,127.0.0.3,127.0.0.4:7001" FrontendHTTPPort="7006" FrontendHTTPSPort="7007" FrontendHost="MyProxy" MulticastAddress="127.0.0.5" MulticastPort="7010" Name="MyCluster"/>

Save your changes and close the config.xml file.

4.2.2 Configuring JMS Resources

In addition to configuring JMS file stores in the Oracle Fusion Middleware Configuration Wizard, proxy services and business services that use JMS require configuration of the following resources:

JMS queues/topics. Oracle Service Bus automatically configures JMS queues for proxy services that are implemented using JMS. You must configure JMS queues/topics for all business services using JMS and for any proxy services that are implemented using non- JMS.

Proxy services can consume messages from a remote queue on a separate domain. In this case, Oracle Service Bus will not create the queue for you. The JMS queues can be created for proxy services only if the queues are on the same local Oracle Service Bus domain.
JMS connection factories. You must configure JMS connection factories for all business services and proxy services implemented using JMS.

For information about configuring JMS resources, see "Configuring Advanced JMS System Resources" in Oracle Fusion Middleware Configuring and Managing JMS for Oracle WebLogic Server.

4.3 Step 3. Configure Oracle Service Bus Security

Oracle Service Bus leverages the security features of Oracle WebLogic Server to ensure message confidentiality and integrity (message-level security), secure connections between clients and Oracle WebLogic Server (transport-level security), and authentication and authorization (access control). For information about the tasks you must complete, see "Security" in the Oracle Fusion Middleware Developer's Guide for Oracle Service Bus.

Note:

You must configure security separately for each Oracle Service Bus domain. Oracle Service Bus does not export or import security configurations.

If you want to configure SSL for your cluster, you can do so when creating your domain or by using the Oracle WebLogic Server Administration Console. For a domain in which security functionality is deployed in a multi-node cluster, you also need to configure keystores, server certificate and private key for each managed server, and so on, for every machine in a cluster. You either need to use a separate keystore for each machine or you can use a single keystore if it is available to all machines.

4.4 Step 4. Starting, Stopping, and Monitoring Managed Servers

This section describes the basic management tasks for the managed servers in your clustered domain:

Section 4.4.1, "Starting and Stopping Managed Servers"
Section 4.4.2, "Monitoring Your Servers"

4.4.1 Starting and Stopping Managed Servers

Node Manager is a utility that enables you to start, stop, and migrate your Oracle WebLogic Server instances. You can start your managed servers using Node Manager in conjunction with the Oracle WebLogic Server Administration Console, or you can create WLST scripts to automate Node Manager functionality.

Tip:

Run the setDomainEnv script before the startNodemanager script to ensure that the Oracle Service Bus classes are available to spawned servers. Alternatively, explicitly set classpath in the Oracle WebLogic Server Administration Console before attempting to start the server.

By default, when the Oracle Fusion Middleware Configuration Wizard generates an Oracle Service Bus cluster domain:

The Domain Singleton Marker Application (domainsingletonmarker.ear) is targeted to the first managed server in the cluster. For data aggregation to function properly, the server that domainsingletonmarker.ear is targeted to must be started first and must be available when other managed servers are started.
The PurgingMDB (msgpurger.ear) is targeted to the first managed server in the cluster. For message purging to function properly, the server where msgpurger.ear is targeted must be available.

For more information on Node Manager, see "Using Node Manager to Control Servers" in Oracle Fusion Middleware Node Manager Administrator's Guide for Oracle WebLogic Server. For a complete overview of methods to start and stop managed servers, see "Starting and Stopping Servers" in Oracle Fusion Middleware Managing Server Startup and Shutdown for Oracle WebLogic Server.

4.4.2 Monitoring Your Servers

Once startup is complete, you can use the Oracle Service Bus Console to verify the status of servers. For information about using Oracle Service Bus Console to monitor your servers, see "Listing and Locating Servers" in "Monitoring" in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

4.5 Step 5. Deploy an Oracle Service Bus Configuration

You deploy an Oracle Service Bus configuration in a clustered environment following the same procedure as for a non-clustered deployment. For a description of the deployment procedure, see Section 2.4, "Step 4. Deploy an Oracle Service Bus Configuration."

Note:

If you have imported a configuration from a non-clustered environment and that configuration includes proxy services that use File, FTP, or Email transports, you must specify a Managed Server for each of those proxy services. The Managed Server list appears in the Oracle Service Bus Console in clustered Oracle Service Bus domains only.

4.6 Step 6. Update Your Domain as Your Production Environment Changes

Production environments change over time and as application use increases. This section describes how to update your domain in response to common production environment change scenarios:

Section 4.6.1, "Adding a Managed Server"
Section 4.6.2, "Dropping a Managed Server"
Section 4.6.3, "Changing a Business Service in a Cluster"
Section 4.6.4, "Installing a New Version of a Proxy Service in a Cluster"

4.6.1 Adding a Managed Server

As use of Oracle Service Bus grows, you can add new managed servers to your Oracle Service Bus cluster to increase capacity. You add a managed server to the cluster using the Oracle WebLogic Server Administration Console. Depending on your configuration, you may also need to perform administrative tasks using the Oracle Service Bus Console.

4.6.1.1 Adding A Managed Server to an Oracle Service Bus Cluster

To add a new managed server to your Oracle Service Bus cluster, perform the steps below using the Oracle WebLogic Server Administration Console:

Verify that the managed server that you want to add to the Oracle Service Bus cluster is not running. Stop the server, if necessary. For information on how to stop a managed server, see "Starting and Stopping Servers" in Oracle Fusion Middleware Managing Server Startup and Shutdown for Oracle WebLogic Server.
If you have not already done so, in the Change Center, click Lock & Edit.
Expand Environment, and select Clusters. On the Summary of Clusters panel, change the cluster address to include the address of the new server.
Expand Services, and select Persistent Stores. Define a new FileStore on the new server, and assign it to a non-migratable server.

Note:
Be sure to create the directory that you specify in the FileStore definition on the new server.
Expand Services and Messaging, and then select JMS Servers. Create a new JMS server on the new server using the new FileStore, and target it to the same non-migratable server.
On the Summary of JMS Servers panel, click the name of the new JMS server. Select jmsResources in the Module Containing Temporary Template list, enter TemporaryTmplt in the Temporary Template Name box, and click Save.
Create the necessary queues and add them to the corresponding distributed destinations by performing the following steps:
1. If you have not already done so, expand Services > Messaging.
2. Select JMS Modules, and click jmsResources.
3. Create a new queue named QueueIn_auto_x, accept the default value for Template, and target the queue to the new server.
  
  Note:
  For the names of queues specified in these steps, x is the number of managed servers currently in the Oracle Service Bus cluster incremented by 1. For example, if you were adding a managed server to a cluster that currently contains two managed servers, then x equals 3. You would be creating QueueIn_auto_3 in this step, and the names of the queues you create in the next steps would also end with 3.
  
  While defining QueueIn_auto_x, create a new subdeployment (for example, sub_new), and then select the JMS server on the new server as the target.
  
  Note:
  You will reuse this subdeployment in the following steps instead of creating a new subdeployment for each queue.
4. Create wli.reporting.jmsprovider.queue_auto_x, and target it to the new server.
5. Create wlsb.internal.transport.task.queue.email_auto_x, and target it to the new server.
6. Create wlsb.internal.transport.task.queue.file_auto_x, and target it to the new server.
7. Create wlsb.internal.transport.task.queue.ftp_auto_x, and target it to the new server.
8. Create wli.reporting.jmsprovider_error.queue_auto_x, and target it to the new server.
9. Create wlsb.internal.transport.task.queue.sftp_auto_x, and target it to the new server.
Expand Services, Messaging, and JMS Modules, and then select the jmsResources module. Add the queues that you created in the previous steps to their corresponding distributed destinations:
1. Add QueueIn_auto_x to dist_QueueIn_auto.
2. Add wli.reporting.jmsprovider.queue_auto_x to dist_wli.reporting.jmsprovider.queue_auto.
3. Add wlsb.internal.transport.task.queue.email_auto_x to dist_wlsb.internal.transport.task.queue.email_auto.
4. Add wlsb.internal.transport.task.queue.file_auto_x to dist_wlsb.internal.transport.task.queue.file_auto.
5. Add wlsb.internal.transport.task.queue.ftp_auto_x to dist_wlsb.internal.transport.task.queue.ftp_auto.
6. Add wli.reporting.jmsprovider_error.queue_auto_x to dist_wli.reporting.jmsprovider_error.queue_auto.
7. Add wlsb.internal.transport.task.queue.sftp_auto_x to dist_wlsb.internal.transport.task.queue.sftp_auto.
In the Change Center, click Activate.
You can now start the new managed server. For information on how to start a managed server, see "Starting and Stopping Servers" in Oracle Fusion Middleware Managing Server Startup and Shutdown for Oracle WebLogic Server.
If your cluster has an HTTP load balancer (software or hardware), add the new managed server to the server list of the load balancer. For example, if you are using the WebLogic HttpClusterServlet, you should add the new server to the web.xml file for the corresponding application. For more information, see "Load Balancing in a Cluster" in Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

4.6.1.2 Updating Business Service Configurations for an Expanded Cluster

If your Oracle Service Bus configuration includes one or more business services that use JMS request/response functionality, then you must also perform the following procedure using the Oracle Service Bus Console after adding the new managed server to the cluster:

In the Change Center, click Create to create a session.
Using the Project Explorer, locate and select a business service that uses JMS request/response. Business services of this type display Messaging Service as their Service Type.
At the bottom of the View Details page, click Edit.
If there is a cluster address in the endpoint URI, add the new server to the cluster address.
On the Edit a Business Service - Summary page, click Save.
Repeat the previous steps for each remaining business service that uses JMS request/response.
In the Change Center, click Activate.

The business services are now configured for operation in the extended domain.

Note:

For business services that use a JMS MesageID correlation scheme, you must edit the connection factory settings to add an entry to the table mapping managed servers to queues. For information on how to configure queues and topic destinations, see "JMS Server Targeting" in Oracle Fusion Middleware Configuring and Managing JMS for Oracle WebLogic Server.

4.6.1.3 Updating Proxy Service Configurations for an Expanded Cluster

If your Oracle Service Bus configuration includes one or more proxy services that use JMS endpoints with cluster addresses, then you must also perform the following procedure using the Oracle Service Bus Console after adding the new managed server to the cluster:

In the Change Center, click Create to create a session.
Using the Project Explorer, locate and select a proxy service that uses JMS endpoints with cluster addresses.
At the bottom of the View Details page, click Edit.
If there is a cluster address in the endpoint URI, add the new server to the cluster address.
On the Edit a Proxy Service - Summary page, click Save.
Repeat the previous steps for each remaining proxy service that uses JMS endpoints with cluster addresses.
In the Change Center, click Activate.

The proxy services are now configured for operation in the extended domain.

4.6.2 Dropping a Managed Server

Using Oracle WebLogic Server administration tools, you can drop a managed server from your Oracle Service Bus cluster. Before deciding to drop a managed server, you should take into account the following considerations:

If your Oracle Service Bus configuration includes one or more proxy services that use File, FTP, or Email transports that have pinned transport pollers to the managed server that you want to remove from the cluster, then you must select a different managed server for each of those proxy services before removing the managed server from the cluster.
The managed server that hosts the Domain Singleton Marker Application must not be dropped from the cluster. If the managed server that hosts the Domain Singleton Marker Application fails, you must perform a manual migration.
If you want to remove the managed server that hosts the Message Reporting Purger application, you must select a different manager server for the Message Reporting Purger and its associated queue (wli.reporting.purge.queue).

For information about dropping a managed server from a cluster, see "Delete Managed Servers" in the Oracle Fusion Middleware Oracle WebLogic Server Administration Console Online Help.

4.6.3 Changing a Business Service in a Cluster

The procedure for changing a business service is the same in both non-clustered and cluster environments. For information about changing a business service, see Section 2.5.1, "Changing a Business Service."

However, the procedure for deploying changes to a business service in a cluster depends on the types of changes made to the business service and the nature of any other changes that might be deployed simultaneously. For more information, see the description of installation strategies in the following section.

4.6.4 Installing a New Version of a Proxy Service in a Cluster

As your business requirements change, you may need to make changes to your proxy services. You can make these changes dynamically online, partially offline, or completely offline. If your changes are backward compatible (that is, you are making no changes to interfaces), you can make your changes dynamically online using the Oracle Service Bus Console. Making other types of changes should be done partially or completely offline, which requires additional system administration steps.

For information about performing online updates, see Section 2.5.3, "Online Configuration Updates."

Making changes that include non-backward compatible changes to proxy service interfaces requires complete offline deployment. To install the new version, follow the procedure below while all servers are operational:

Quiesce all inbound messages.
Confirm all asynchronous backlogged messages have been processed.
Make the necessary changes in the proxy service, and test to verify the proxy service operates as required.
Resume accepting inbound messages.

For more information about backward compatibility and installation strategies, see Section 2.5.2, "Installing a New Version of a Proxy Service."