Oracle Application Server 10g High Availability Guide 10g (9.0.4) Part Number B10495-01 |
|
This section provides instructions to manage your Infrastructure's high availability environment. Instructions for operations such as stopping, starting, and recovering from scheduled and unplanned outages are provided. The two high availability solutions are discussed:
The instructions in this section detail the steps for starting and stopping the OracleAS Infrastructure in an OracleAS Cold Failover Cluster.
Use the following steps to start the Infrastructure in an OracleAS Cold Failover Cluster:
ORACLE_HOME
environment variable to the Infrastructure's Oracle home.
ORACLE_SID
environment variable to the metdata repository's system identifier.
PATH
environment variable to include the Infrastructure's $ORACLE_HOME/bin directory
.
$ORACLE_HOME/bin/lsnrctl start
If OPMN daemon is not running, start both OPMN daemon and OPMN-managed processes:
$ORACLE_HOME/opmn/bin/opmnctl startall
If OPMN daemon is running, start all OPMN-managed processes collectively:
$ORACLE_HOME/opmn/bin/opmnctl startproc
Alternatively, to individually start up OPMN-managed processes:
$ORACLE_HOME/opmn/bin/opmnctl startproc ias-component=HTTP_Server
$ORACLE_HOME/opmn/bin/opmnctl startproc ias-component=OID
$ORACLE_HOME/opmn/bin/opmnctl startproc ias-component=OC4J instancename=OC4J_SECURITY
$ORACLE_HOME/opmn/bin/opmnctl status
$ORACLE_HOME/bin/emctl start iasconsole
Or:
$ORACLE_HOME/bin/emctl startifdown iasconsole
Use the following steps to stop the OracleAS Infrastructure in an OracleAS Cold Failover Cluster:
ORACLE_HOME
environment variable to the Infrastructure's Oracle home.
ORACLE_SID
environment variable to the metdata repository's system identifier.
To shutdown the OPMN daemon and all OPMN-managed processes:
$ORACLE_HOME/opmn/bin/opmnctl stopall
To shutdown all OPMN-managed processes but leave the OPMN daemon running:
$ORACLE_HOME/opmn/bin/opmnctl stopproc
Alternatively, to individually shutdown all OPMN-managed processes:
$ORACLE_HOME/opmn/bin/opmnctl stopproc ias-component=OC4J
instancename=OC4J_SECURITY
$ORACLE_HOME/opmn/bin/opmnctl stopproc ias-component=OID
$ORACLE_HOME/opmn/bin/opmnctl stopproc ias-component=HTTP_Server
$ORACLE_HOME/bin/lsnrctl stop
$ORACLE_HOME/bin/emctl stop iasconsole
Note: In this release of Oracle Application Server 10g (9.0.4), Oracle Application Server Active Failover Cluster is a limited release feature. Check OracleMetalink (http://metalink.oracle.com) for the most current certification status of this feature or consult your Oracle sales representative before deploying this feature in a production environment. |
The instructions in this section detail the steps for starting and stopping the Infrastructure in the OracleAS Active Failover Cluster high availability solution.
For an OracleAS Active Failover Cluster-enabled Infrastructure, each node in the cluster is functionally equivalent to the other nodes. All nodes access a common repository. The database instance and the individual OracleAS processes need to be started on each node of the cluster. At any given time, the load balancer should be configured to direct traffic to only the active nodes. The order of starting up Infrastructure instances on all nodes is:
$ORACLE_HOME/bin/gsd
$ORACLE_HOME/bin/srvctl start -p <database_name>
The global services daemon on each node ensures that the local database processes on each node are started.
If OPMN daemon is not running, start both OPMN daemon and OPMN-managed processes (following command need only be run once):
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2> startall
If OPMN daemon is running, you can start all OPMN-managed processes on all nodes (following command need only be run once):
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2> startproc
For example, assuming there are two nodes in the cluster:
$ORACLE_HOME/opmn/bin/opmnctl
@instance:infra_node1:infra_node2 startproc
Alternatively, to individually start up OPMN-managed processes on all nodes (following commands need only be run once):
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2> startproc
ias-component=HTTP_Server
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2> startproc
ias-component=OID
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2> startproc
ias-component=OC4J instancename=OC4J_SECURITY
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2> status
$ORACLE_HOME/bin/emctl start iasconsole
Or:
$ORACLE_HOME/bin/emctl startifdown iasconsole
The OracleAS Active Failover Cluster-enabled Infrastructure provides better availability since individual Infrastructure instances can be shut down while others continue to be available. The order of shutting down an instance is:
To stop all OPMN-managed processes and OPMN daemons on all nodes in the cluster, run the following command once:
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2> stopall
To stop all OPMN-managed processes but leave the OPMN daemon running, run the following command once:
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2> stopproc
To individually stop OPMN-managed processes on all cluster nodes, run the following commands once:
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2> stopproc
ias-component=OC4J instancename=OC4J_SECURITY
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2> stopproc
ias-component=OID
$ORACLE_HOME/opmn/bin/opmnctl
@instance:<instancename_on_node1>:<instancename_on_node2>
stopproc ias-component=HTTP_Server
$ORACLE_HOME/bin/srvctl stop -p <database_name>
The global services daemon on each node ensures that the local database processes on each node are stopped.
$ORACLE_HOME/bin/emctl stop iasconsole
Monitoring the OracleAS Active Failover Cluster-enabled Infrastructure is similar to monitoring any regular Infrastructure deployment. The only special consideration is monitoring the load balancer and ensuring that it is directing traffic to the active nodes. Please contact your load balancer vendor on monitoring the availability of the load balancer as well as the sanity of its configuration.
The OracleAS Active Failover Cluster-enabled Infrastructure provides continued availability under many scheduled and unplanned outages. The outages handled automatically by this solution are listed in Table 5-1 below.
For the outages in Table 5-2, there may be a small downtime. Having a disaster recovery site can mitigate but not eliminate this downtime. A standby site can be activated while the production site experiences an outage. Refer to the section "Oracle Application Server Disaster Recovery Solution" for more information.
Outage Type | Outages |
---|---|
Scheduled |
Database maintenance |
Unplanned |
Fault tolerant load balancer complete failure |
System behavior under these outages is as follows:
Restoration of resiliency post-outage involves the addition of an Oracle Application Server instance to the current set of active Oracle Application Server instances. The primary steps involved are:
Each node of an OracleAS Active Failover Cluster has in its file system configuration files that are part of the Infrastructure but are not stored in the OracleAS Metadata Repository. These files are likely to change as administration operations such as the following are performed on each node:
A primary requirement for an OracleAS Active Failover Cluster deployment is that all nodes in the cluster are configured similarly. The configuration files should be similar, if not, identical.
In order to maintain a consistent Infrastructure across all nodes in the OracleAS Active Failover Cluster, a command line utility is provided to synchronize these configuration files across the nodes. This utility is called Oracle Application Server Active Failover Cluster Runtime Control Utility and can be invoked using the command afcctl
. Synchronization of files using this utility should be performed at least everytime an administration change is made to Oracle Application Server.
afcctl
This section describes how to download and install the afcctl
utility and perform initial configuration. The overall steps are:
afcctl
Utility
The afcctl
utility is available with the utility CD that comes with your Oracle Application Server product. The file containing the utility is <mount-point>
/utilities/ha/afcctl.zip
(where <mount-point>
is the mount point of the CD-ROM drive. This file is installed along with the Oracle Application Server Backup and Recovery Tool. Hence, you need to install the latter tool first. Refer to Oracle Application Server 10g Administrator's Guide for instructions on installing this tool.
After the Oracle Application Server Backup and Recovery Tool has been installed on a node, create the directory <ORACLE_HOME>
/afcctl/
on that node, and copy afcctl.zip
to the new directory. Do this for every node in the OracleAS Active Failover Cluster.
afcctl
Utility
<ORACLE_HOME>
/afcctl/
directory and unzip afcctl.zip
. It should contain the following files:
afcctl afcctl.pl afcctl.jar afcctl_exclude.inp README
> chmod 755 afcctl
afcctl
utility relies on the Oracle Application Server Backup & Recovery Tool being available and installed. Make sure that the latter tool has been installed and the .inp
files of the tool are accessible by the afcctl
utility.
afcctl
Run the afcctl
utility on any node in the OracleAS Active Failover Cluster to perform the following tasks:
Immediately after installation of OracleAS and afcctl
, you should set the default configuration timestamp across the OracleAS Active Failover Cluster. This baseline timestamp marks the default configuration after installation.
After this baseline is set, the next synchronization performed using afcctl
with the sync
option synchronizes only those configuration files that have changed since the baseline and the time the afcctl sync
command is run.
To create a timestamp baseline, use the following command:
afcctl createbase -p <dbname>|-r <host1>,<host2>,...,<hostN> [-c <cp_exec>]
where:
<dbname>
is the name of the Infrastructure database
<host1>
,
<host2>
,..,
<hostN>
is a comma separated list of remote hosts in the OracleAS Active Failover Cluster
<cp_exec>
is the full path to a remote copy utility to be used to copy files from the current node to other nodes in the cluster. By default, afcctl
uses /usr/bin/rcp
, or /usr/local/bin/scp
if the former (rcp
) is not found. If neither of these are present or if you wish scp
to precede rcp
in the invocation order, use the -c
<cp_exec>
option to specify the copy utility to be used.
After the initial configuration baseline is set using the createbase
option, you can synchronize any configuration changes across the cluster using the sync
option. This option synchronizes changed configuration by copying only the modified configuration files from the current node to all nodes in the OracleAS Active Failover Cluster.
The command syntax for invoking a synchronization is:
afcctl sync -p <dbname>|-r <host1>,<host2>,...,<hostN> -f <filename>|<file_list_dir>
[-c <cp_exec>] [-l <hostname>]
where :
<dbname>
is the name of the Infrastructure database
<host1>
,
<host2>
,..,
<hostN>
is a comma separated list of remote hosts in the OracleAS Active Failover Cluster
<filename>
is the name of the file to be synchronized
<file_list_dir>
is the name of the directory where the .inp
files reside
<cp_exec>
is the full path to a remote copy utility to be used to copy files from the current node to other nodes in the cluster. By default, afcctl
uses /usr/bin/rcp
, or /usr/local/bin/scp
if the former (rcp
) is not found. If neither of these are present or if you wish scp
to precede rcp
in the invocation order, use the -c
<cp_exec>
option to specify the copy utility to be used.
<hostname>
is the hostname of the local host at installation time of the Infrastructure
Take note of the following when using the above command line:
-p
option can only be used if the gsd process has been started and is running. This option also requires that the Infrastructure database and its instances have been registered with the srvm repository, which is the case by default. The -p
option automatically determines the nodes of the OracleAS Active Failover Cluster deployment and propagates the necessary files to the other nodes.
-r
option can be used in lieu of -p
to explicitly specify the hostnames of the nodes that need to be synchronized with the node the utility is run on. No other process dependencies exist in this case. Ensure that the hostname(s) specified are valid nodes of the OracleAS Active Failover Cluster deployment.
<file_list_dir>
for the -f
option is the directory where the .inp
files of the Oracle Application Server Backup and Recovery Tool exist. The afcctl
utility references these files for the list of configuration files that need to be synchronized.
<filename>
for the -f
option is used only when a single file needs to be synchronized across the cluster.
afcctl
, set up user equivalence so that the rcp
and scp
copy utilities can be used without further authentication for the remote hosts. User equivalence is also required for the OracleAS Active Failover Cluster installation. Refer to the high availability chapter in Oracle Application Server 10g Installation Guide for instructions on how to set up user equivalence.
-l
is optional. It is used to specify the host where afcctl
is run and should be specified only in cases where the local hostname during Infrastructure installation is different from the default hostname.
To find out which configuration files on a node in the OracleAS Active Failover Cluster has changed since the last synchronization, use the following command line syntax:
afcctl list -f <filename>|<file_list_dir>
where
<filename>
is the name of a file that is to be checked for any updates since the last synchronization.
<file_list_dir>
is the name of the directory where the .inp
files of the Oracle Application Server Backup and Recovery Tool exist. Files in that directory which have been changed since the last synchronization are listed.
A text file containing a list of files that have changed since the last synchronization is created in the /tmp
directory. See an example in the section "Example".
The syntax above can be used on any node of the OracleAS Active Failover Cluster deployment. It displays the files that have changed, since the last synchronization, on the node it is executed on. The returned list of files can be different depending on the site. To synchronize the listed file(s) individually, the -f
<filename>
option of the afcctl
sync
command can be used after determing which version of the file is the latest.
Oracle recommends that all nodes are configured similarly. If, however, some configuration files need to be different, their names can be added to the exclude file, afcctl_exclude.inp
, so that they are not synchronized across the cluster when afcctl
is run. afcctl_exclude.inp
is found in the same directory where you uncompressed afcctl.zip
.
After any administrative operation to Oracle Application Server (through Application Server Control or DCM), which can change any of the configuration files, do the following on each node of the OracleAS Active Failover Cluster:
ORACLE_HOME
environment variable. For example, in a Bourne shell environment, type:
$ export ORACLE_HOME=/home/oracle/test1
afcctl
with the list
option to display the configuration files that have changed on the current node.
$ afcctl list -f ./br_inp_dir Oracle Application Server Active Failover Cluster Run Time Control Utility Copyright (c) 2002, 2003 Oracle Corporation. All rights reserved. Last Sync up time was Mon Sep 8 11:09:11 2003 Check the following for list of files that have changed since last sync work/Files_to_Change_and_Copy.23123 Please look at log/afcctl.log for more information. Exiting....
The file work/Files_to_Change_and_Copy.23123
is created to contain the list of configuration files that have changed since the last synchronization.
$ cat work/Files_to_Change_and_Copy.23123 /home/oracle/test1/Apache/Apache/conf/ssl.wlt/default/ewallet.p12 /home/oracle/test1/ldap/admin/oidpwdlldap1 /home/oracle/test1/ldap/admin/oidpwdrgit11
afcctl
with the sync
option to synchronize files from one node in the OracleAS Active Failover Cluster to another.
$ afcctl sync -r hasun26 -f ./backup_scripts Oracle Application Server Active Failover Cluster Run Time Control Utility Copyright (c) 2002, 2003 Oracle Corporation. All rights reserved. Files to massage & copy are listed in work/Files_to_Change_and_Copy.22339 Files to copy are listed in work/Files_to_Copy.22339 Do you want to sync up files from hasun25.us.oracle.com to hasun26.us.oracle.com (y/n) ? y Syncing up files ............................................................................ ...........................................! Syncing completed Do you want to update the dcm repository with configuration files from "hasun25.us.oracle.com" (y/n) ? y DCM update repository started DCM update repository completed Please look at log/afcctl.log for more information. Exiting....
afcctl
dcmctl
) to make any changes to the Infrastructure configuration.
If you have to make changes manually, designate a node as the administration node and perform all changes on it. Then, run the afcctl
tool from this node to propagate the changes to other nodes in the OracleAS Active Failover Cluster.
list
option regularly on all nodes of the cluster to verify that nothing has been changed locally since the last synchronization. Reconcile any changes across the nodes, if required.
.inp
files with the Oracle Application Server Backup & Recovery Tool, keep in mind that any changes to the exclude and personalization files of the tool impacts the afcctl
utility as well. See the Oracle Application Server 10g Administrator's Guide for more information on the tool.
|
![]() Copyright © 2003 Oracle Corporation. All Rights Reserved. |
|