9 High Availability

Requirements for High Availability

• Use two or more systems of the same model and configured identically:

  • Processor class

  • Operating system

  • Oracle Enterprise Manager Ops Center software version, including updates

  • Network interfaces that are cabled identically to the same subnets

• Add an asset tag to identify the active Enterprise Controller and to distinguish it from the standby Enterprise Controller. You can add a tag by using the Edit Asset action.

• Maintain the standby Enterprise Controller's system in the same way as the active Enterprise Controller. The active and standby Enterprise Controllers must use the same version of Oracle Enterprise Manager Ops Center software. If you cannot use the user interface to verify the installed software versions at the time that you need to transfer functions to the standby system, view the content of the /n1gc-setup/.version.properties file. The product.version property lists the specific revision level of the installed software. For example:

  # cat /n1gc-setup/.version.properties
  #Note: This file is created at build time.
  #Wed Jun 30 15:28:45 PDT 2010
  version=dev-ga
  date=2010/06/30 15\:28
  build.variation=xvmopscenter
  product.version=2.6.0.1395
  product.installLocation=/var/opt/sun/xvm/EnterpriseController_installer_2.6.0.1395
  #
  

  Verify that the product.version property lists the same version on the active and standby Enterprise Controllers before you perform a relocate procedure.

Limitations

• User accounts and data that are not associated with Oracle Enterprise Manager Ops Center are not part of the relocate process. Only Oracle Enterprise Manager Ops Center data is moved between the active and standby Enterprise Controllers.

• UI sessions are lost on relocate.

• The EC HA configuration applies only to the Enterprise Controller and its co-located Proxy Controller and not to other standalone Proxy Controllers.

Preparing for High Availability with Oracle Clusterware

Installing and configuring Oracle Clusterware is the first step in setting up High Availability in your environment.

Installing Oracle Clusterware

1. Install Oracle Clusterware in your environment. The Oracle Clusterware installation documentation is available at http://download.oracle.com/docs/cd/B28359_01/install.111/b28262/toc.htm for Oracle Solaris systems and at http://download.oracle.com/docs/cd/B28359_01/install.111/b28263/toc.htm for Linux systems.

2. If you are using a local database, switch to a remote database using the procedure in the Database Management chapter.

Making the Current Enterprise Controller the Active Node

Once your environment is prepared, configure the current Enterprise Controller as the active node.

To Make the Current Enterprise Controller the Active Node

1. Stop the Enterprise Controller using the ecadm command and the stop subcommand.

2. Use the ecadm command with the ha-configure-primary subcommand to configure the system as the active Enterprise Controller.

   For example:

   # ./ecadm ha-configure-primary
   INFO: HAECClusterwareAdapter/doConfigurePrimary() Stopping Ops Center ...
   INFO: HAECClusterwareAdapter/doConfigurePrimary() Ops Center stopped
   INFO: HAECClusterwareAdapter/createActionScript() created Resource Action Script='/var/opt/sun/xvm/ha/EnterpriseController'
   INFO: HAECClusterwareAdapter/doConfigurePrimary() created Clusterware Action Script='/var/opt/sun/xvm/ha/EnterpriseController'
   INFO: HAECClusterwareAdapter/doConfigurePrimary() created Clusterware Resource='EnterpriseController'
   INFO: HAECClusterwareAdapter/doHAStart() starting resource='EnterpriseController' on node='primary-system'
   INFO: HAECClusterwareAdapter/doHAStart()statusSB='CRS-2672: Attempting to start 'EnterpriseController' on 'primary-system'
   CRS-2676: Start of 'EnterpriseController' on 'primary-system' succeeded'
   INFO: HAECClusterwareAdapter/doHAStart() started resource='EnterpriseController' on node='primary-system'
   INFO: HAECClusterwareAdapter/doConfigurePrimary() Ops Center started on node='primary-system'
   ecadm:    --- Enterprise Controller successfully configured HA primary node
   # 
   

Installing a Standby Node

Once you have configured one Enterprise Controller as the active node, you can install and configure standby nodes.

To Install the Enterprise Controller on a Standby Node

1. If you are installing on Oracle Solaris 11, and if the system requires an HTTP proxy to reach the Internet, set the http_proxy and https_proxy environment variables using the following format:

   • http_proxy: <protocol>://<host>:<port> - This variable specifies the proxy server to use for HTTP.

   • https_proxy: <protocol>://<host>:<port> - This variable specifies the proxy server to use for HTTPS.

2. Create a temporary directory on your system, then copy or move the appropriate Oracle Enterprise Manager Ops Center archive for your system from delivery media to the temporary directory that you created. For example:

   # mkdir /var/tmp/OC
   # cp enterprise-controller.SunOS.i386.12.1.0.2001.tar.zip /var/tmp/OC
   

   The installation archive consumes about 3.5 GBytes of disk space.

3. Change to the directory where the installation archive is located on your system.

   # cd /var/tmp/OC
   #
   

4. Expand the installation archive, then list the contents of the expanded directory.

   • If your installation archive has the .tar.zip extension, use the unzip and tar commands to uncompress and un-tar the archive, then list the contents of the temporary directory.

     For example:

     # gzcat enterprise-controller.Solaris.i386.12.1.0.2001.tar.zip | tar xf -
     # ls
     enterprise-controller.Solaris.i386.12.1.0.2001.tar.gz
     xvmoc_full_bundle
     #
     

   • If your installation archive has the .zip extension, use the unzip command to uncompress the archive. For example:

     # unzip enterprise-controller.Solaris.i386.12.1.0.2001.zip
     # ls
     enterprise-controller.Solaris.i386.12.1.0.2001.zip
     xvmoc_full_bundle
     #
     

5. Create a remote database properties file on the Enterprise Controller system. The remote database properties file must contain the location of the remote database and a user name and password that can access the database.

   For example:

   # vi /var/tmp/OC/DB/RemoteDBProps.txt
   mgmtdb.appuser=user
   mgmtdb.password=userpass
   mgmtdb.roappuser=user
   mgmtdb.ropassword=userpass
   mgmtdb.dburl=jdbc:oracle:thin:@<database hostname>:<port>/<database service name>
   

6. Change directory to xvmoc_full_bundle, and run the install script with the --remoteDBprops <path to remote database properties file> and --StandbyEC options. Each installation archive only contains an install script that is appropriate for its intended OS and platform. For example:

   # cd xvmoc_full_bundle
   # ./install -s --remoteDBprops=/<Path to remote database properties file>/remoteDbCreds.txt --StandbyEC
   

7. The Oracle Configuration Manager installation text is displayed. Enter the My Oracle Support user name or email address that you want to associate with Oracle Enterprise Manager Ops Center.

   Provide your email address to be informed of security issues, install and
   initiate Oracle Configuration Manager. Easier for you if you use your My
   Oracle Support Email address/User Name.
   Visit http://www.oracle.com/support/policies.html for details.
   Email address/User Name:
   

8. If you want security updates to appear on your My Oracle Support page, enter your My Oracle Support password.

   Provide your My Oracle Support password to receive security updates via your My Oracle Support account.
   Password (optional):
   

   The screen clears, then the install script displays a list of installation tasks that automatically updates as the installation proceeds. For example:

   Ops Center Enterprise Controller Installer
   (version 12.1.0.0 on SunOS)
   
   1. Check for installation prerequisites. [Completed]
   2. Configure file systems. [Completed]
   3. Install prerequisite packages. [Not Completed]
   4. Install Agent components. [Not Completed]
   5. Create Deployable Proxy Bundles. [Not Completed]
   6. Install application packages. [Not Completed]
   7. Run postinstall tasks. [Not Completed]
   8. Install Expect. [Not Completed]
   9. Install IPMI tool. [Not Completed]
   10. Set database credentials. [Not Completed]
   11. Install and configure Oracle Database. [Not Completed]
   12. Install Service container components. [Not Completed]
   13. Install Core Channel components. [Not Completed]
   14. Install Proxy Core components. [Not Completed]
   ..........................
   19. Initialize and start services. [Not Completed]
   (2 of 19 Completed)
   
   Executing current step: Install prerequisite packages...
   

9. Review and correct any problems when the install script checks for installation prerequisites that are not met. For example, this install script detected insufficient disk space:

   Warning for Step: Check for installation prerequisites.
   The following is a portion of the installer
   log which may indicate the cause of the warning.
   If this does not indicate the cause of the
   warning, you will need to view the full log
   file. More information on how to do that is
   available below.
   You may choose to ignore this warning by selecting to continue.
   * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
   
   Ignoring job: 01checkRPMs.pl
   Ignoring job: 03removeEmptyDirs.pl
   
   
   Executing job: jobs/00checkPrereqs.pl --install
   
   WARNING: Installation prerequisites not met:
   Disk: / 72G needed, 24G available.
   * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
   Please fix the problem and then try this step again.
   For a full log of the failed install see the file: /var/tmp/installer.log.9361.
   
   t. Try this step again (correct the failure before proceeding)
   c. Continue (ignore the warning)
   x. Exit
   Enter selection: (t/c/x)
   

   You can enter t to try again, c to continue and ignore the warning, or x to exit the install script. Typically you would exit the install script, correct the problem, and then run the install script again, which resumes from where it stopped. Choose to continue and ignore the warning only if you accept the impact that the error condition will have on your installation. Entering t typically produces the same error, unless you are able to correct the problem before trying the step again. If the install script finds that all prerequisites have been satisfied, or if you choose to continue despite the warning, the install script continues and installs all Enterprise Controller and Proxy Controller components.

   When complete, the install script displays a confirmation that all components have been installed. The /var/tmp/installer.log.latest file contains the installation log.

10. Create a password file containing the root user name and password for the active Enterprise Controller. For example:

    # touch /tmp/creds.props
    # chmod 400 /tmp/creds.props
    # vi /tmp/creds.props
    # cat /tmp/creds.props
    username:root
    password:XXXXX
    

11. Use the ecadm command with the ha-configure-standby and -p <password file> subcommands to configure the node as a standby node.

    For example:

    # ecadm ha-configure-standby -p /tmp/creds.props
    INFO: HAECClusterwareAdapter/doConfigureStandby() Stopping Ops Center ...
    INFO: HAECClusterwareAdapter/doConfigureStandby() Ops Center stopped
    INFO: remoteFileCopy() copied '/etc/passwd' from remoteHostname='primary-system' to local file='/tmp/activeNodepw'
    <output omitted>
    ecadm:    --- Enterprise Controller successfully configured HA standby node
    

12. Use the ecadm command with the ha-status -d option to check the status of the standby Enterprise Controller.

    For example:

    # ecadm ha-status -d
    INFO: HAECClusterwareAdapter/doHAStatus() Status:
    # HAEC Cluster Info: Thu Sep 29 15:49:09 MDT 2011
    haec.cluster.active.node=primary
    haec.cluster.nodes=standby, primary
    haec.ec.public.nics=nge1
    haec.ec.status=ONLINE
    <output omitted>
    haec.cluster.script=/var/opt/sun/xvm/ha/EnterpriseController
    haec.cluster.crsctl=/u01/app/11.2.0/grid/bin/crsctl
    # End of Cluster Info
    ecadm:    --- Enterprise Controller ha-status command succeeded
    Status stored in file: /var/opt/sun/xvm/ha/HAECStatus
    # 
    

Adding a Network Resource

You can add a network resource using the crsctl command.

To add a network resource, run the crsctl add resource command with the following format:

/u01/app/11.2.0/grid/bin/crsctl add resource <resource name> -type application -attr ACTION_SCRIPT=/u01/app/11.2.0/grid/bin/usrvip, USR_ORA_NETMASK=<netmask>,USR_ORA_VIP=<vip IP address>,USR_ORA_START_TIMEOUT=0,USR_ORA_STOP_TIMEOUT=0,USR_ORA_STOP_MODE=immediate,USR_ORA_IF=<network interface>,USR_ORA_OPI=false,USR_ORA_CHECK_TIMEOUT=0,USR_ORA_DISCONNECT=false,USR_ORA_PRECONNECT=none,HOSTING_MEMBERS=<node1>:<node2>

The following options are included in this format:

• <resource name> - Specifies the resource name.

• -type application

• USR_ORA_IF=<network interface> - Specifies the network interface (NIC) for the network resource.

• USR_ORA_VIP= <ipaddress> - Specifies the IP address for the network resource.

• USR_ORA_NETMASK=<netmask> - Specifies the netmask for the network resource.

• USR_ORA_IF=<network interface> - Specifies the network interface (NIC) for the network resource.

• HOSTING_MEMBERS=<node1>:<node2> - Specifies the cluster nodes hosting the Enterprise Controller.

• ACTION_SCRIPT=/u01/app/11.2.0/grid/bin/usrvip

• PLACEMENT=favored

Modifying a Network Resource

You can modify an existing network resource using the crsctl command.

To add a network resource, run the crsctl modify resource command with the following format:

./crsctl modify resource <resource name> -attr <attribute>=<new value>, <attribute>=<new value>,...

The following attributes can be modified:

• USR_ORA_IF=<network interface> - Specifies the network interface (NIC) for the network resource.

• USR_ORA_VIP= <ipaddress> - Specifies the IP address for the network resource.

• USR_ORA_NETMASK=<netmask> - Specifies the netmask for the network resource.

• USR_ORA_IF=<network interface> - Specifies the network interface (NIC) for the network resource.

• HOSTING_MEMBERS=<node1>:<node2> - Specifies the cluster nodes hosting the Enterprise Controller.

• ACTION_SCRIPT=/u01/app/11.2.0/grid/bin/usrvip

• PLACEMENT=favored