Skip Headers

Oracle® Real Application Clusters Installation and Configuration Guide
10g Release 1 (10.1) for AIX-Based Systems, hp HP-UX PA-RISC (64-bit), hp Tru64 UNIX, Linux, Solaris Operating System (SPARC 64-bit), and Windows (32-bit) Platforms

Part Number B10766-02
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Feedback

Go to previous page
Previous
Go to next page
Next
View PDF

3 Pre-Installation Tasks for Installing RAC on HP-UX Systems

This chapter describes the tasks that you must complete before you start the Oracle Universal Installer. It includes information about the following tasks:

Log In to the System as root

Before you install the Oracle software, you must complete several tasks as the root user. To log in as the root user, complete one of the following procedures:


Note:

Unless you intend to complete a silent installation, you must install the software from an X Window System workstation, an X terminal, or a PC or other system with X server software installed.

For more information about non-interactive and silent installations, see the appendices in Oracle Database Installation Guide for UNIX Systems.

Check the Hardware Requirements

The system must meet the following minimum hardware requirements:

To ensure that the system meets these requirements, follow these steps:

  1. To determine the physical RAM size, enter the following command:

    # /usr/sbin/dmesg | grep "Physical:"

    If the size of the physical RAM installed in the system is less than the required size, you must install more memory before continuing.

  2. To determine the size of the configured swap space, enter the following command:

    # /usr/sbin/swapinfo -a

    If necessary, see your operating system documentation for information about how to configure additional swap space.

  3. To determine the amount of disk space available in the /tmp directory, enter the following command:

    # bdf /tmp

    If there is less than 400 MB of disk space available in the /tmp directory, complete one of the following steps:

    • Delete unnecessary files from the /tmp directory to achieve the required disk space.

    • Set the TEMP and TMPDIR environment variables when setting the oracle user's environment (described later).

    • Extend the file system that contains the /tmp directory. If necessary, contact your system administrator for information about extending file systems.

  4. To determine the amount of free disk space on the system, enter the following command:

    # bdf

  5. To determine whether the system architecture can run the software, enter the command:

    # /bin/getconf KERNEL_BITS

    The expected output is the value 64.


    Note:

    If you do not see the expected output, you cannot install the software on this system.

Check the Network Requirements

Check that you have the networking hardware and internet protocol (IP) addresses required for an Oracle Real Application Clusters installation.


Note:

For the most up-to-date information about supported network protocols and hardware for RAC installations, see the Certify pages on the OracleMetaLink Web site:
http://metalink.oracle.com

Network Hardware Requirements

Each node in the cluster must meet the following requirements:

  • Each node must have at least two network adapters; one for the public network interface and one for the private network interface (the interconnect).

  • The interface names associated with the network adapters for each network must be the same on all nodes.

  • For increased reliability, you can configure redundant public and private network adapters for each node.

  • For the public network, each network adapter must support TCP/IP.

  • For the private network, Oracle supports the following interconnect protocols and hardware:

    • User datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP (Gigabit Ethernet or better recommended)


      Note:

      UDP is the default interconnect protocol for RAC and TCP is the interconnect protocol for Oracle CRS.
    • Hyper Messaging protocol (HMP) using HyperFabric adapters and switches

IP Address Requirements

Before starting the installation, you must identify or obtain the following IP addresses for each node:

  • An IP address and an associated host name registered in the domain name service (DNS) for each public network interface

  • One unused virtual IP address and an associated virtual host name registered in DNS that you will configure for the primary public network interface

    The virtual IP address must be in the same subnet as the associated public interface. After installation, you can configure clients to use the virtual host name or IP address. If a node fails, its virtual IP address fails over to another node.

  • A private IP address and optional host name for each private interface

    Oracle recommends that you use private network IP addresses for these interfaces, for example: 10.*.*.* or 192.168.*.*. You can use the /etc/hosts file on each node to associate private host names with private IP addresses.

For example, if each node has two public and two private interfaces, you might have the following host names and IP addresses on one of the nodes (rac1) and similar host names and IP addresses on the other nodes:

Host Name Type IP Address Registered In
rac1.mydomain.com Public 143.47.43.100 DNS
rac1-2.mydomain.com Public 143.46.51.101 DNS
rac1-vip.mydomain.com Virtual 143.46.43.104 DNS
rac1-priv1 Private 10.0.0.1 /etc/hosts
rac1-priv2 Private 10.0.0.2 /etc/hosts

Checking the Network Requirements

To verify that each node meets the requirements, follow these steps:

  1. If necessary, install the network adapters for the public and private networks and configure them with either public or private IP addresses.

  2. Register the host names and IP addresses for the public network interfaces in DNS.

  3. For each node, register one virtual host name and IP address in DNS.

  4. For each private interface on every node, add a line similar to the following to the /etc/hosts file on all nodes, specifying the private IP address and associated private host name:

    10.0.0.1     rac1-priv1
    
    
  5. To identify the interface name and associated IP address for every network adapter, enter the following command:

    # /usr/bin/netstat -in
    
    

    From the output, identify the interface name and IP address for all network adapters that you want to specify as public or private network interfaces.


    Note:

    When you install Oracle CRS and RAC, you will require this information.

Check the Software Requirements

See the following sections for information about checking the software requirements:


Note:

The Oracle Universal Installer performs checks on your system to verify that it meets the requirements listed for HP-UX. To ensure that these checks pass, verify the requirements before you start the Installer.

Check for Required Software

Depending on the products that you intend to install, verify that the following software is installed on the system. The procedure following the table describes how to check these requirements.

Installation Type or Product Requirement
All installations Operating system version:

HP-UX 11i (11.11) PA-RISC

Oracle Spatial HP-UX Developer's Toolkit - X11, Motif, and Imake:
X11MotifDevKit

Note: This software is required only to build the sample programs.

Real Application Clusters
  • HP Serviceguard A11.15.00 or higher
  • HP Serviceguard Extension for RAC A11.15.00 or higher

  • HyperFabric software - Required if you want to use Hyper Messaging protocol as the cluster interconnect

  • .01 or higher

Note: HP Serviceguard is required only if you want to use HP's shared logical volumes for CRS or database files.

Oracle Messaging Gateways IBM WebSphere MQ V5.3, client and server:
MQSERIES.MQM-CL-HPUX
MQSERIES.MQM-SERVER

TIBCO Rendezvous 7.2

Pro*C/C++, Oracle Call Interface, Oracle C++ Call Interface, Oracle XML Developer's Kit (XDK) HP C/ANSI C Compiler (B.11.11.08 or higher):
C-ANSI-C

HP aC++ Compiler (A.03.50 or higher):

ACXX
Oracle JDBC/OCI Drivers You can use the following optional Java SDK versions with the Oracle JDBC/OCI drivers, however they are not required for the installation:
  • Java SDK 1.3.1.02 with the JNDI extension

  • Java SDK 1.2.2.09

Note: Java SDK 1.4.2.00 is installed with this release.

Oracle DCE Integration (part of Oracle Advanced Security) or Oracle Net protocol support for DCE DCE (B.11.11.10 or higher):
DCE-C-Tools
DCE-CDS-Server
DCE-Core
DCE-CoreAdmin
DCE-CoreTools
DCE-SEC-Server
Oracle Transparent Gateway for Sybase Either of the following:
  • Sybase Adaptive Server Enterprise V12 or 12.5

  • Sybase Open Client V12 or 12.5

    Open Client is required if the Sybase server is not on the local node.

Oracle Transparent Gateway for Informix One of the following:
  • Informix Dynamic Server v7.3, 7.4, or 9.*

  • Informix ESQL/C v7.3, 7.4, or 9.*

    ESQL/C is required if the Informix server is not on the local node.

Note: This version of the gateway does not support Informix SE.

Oracle Transparent Gateway for Ingress Ingres II version 2.0, 2.5, or 2.6
Oracle Transparent Gateway for Teradata NCR Teradata ODBC Driver v02.04.00.00

To ensure that the system meets these requirements, follow these steps:

  1. To determine which version of HP-UX is installed, enter the following command:

    # uname -a
    HP-UX hostname B.11.11 U 9000/800 109444686 unlimited-user license
    
    

    In this example, the version of HP-UX 11i is 11.11.

  2. To determine whether a bundle, product, or file set is installed, enter a command similar to the following, where level is bundle, product, or fileset:

    # /usr/sbin/swlist -l level | more
    
    

    If a required bundle, product, or file set is not installed, you must install it. See your operating system or software documentation for information about installing products.

Check for Required Patches

Depending on the products that you intend to install, verify that the following patches are installed on the system. The procedure following the table describes how to check these requirements.


Note:

There may be more recent versions of the patches listed installed on the system. If a listed patch is not installed, determine whether a more recent version is installed before installing the version listed.
Installation Type or Product Requirement
All installations Quality Pack bundle:

HP-UX 11i Quality Pack (GOLDQPK11i), June 2003 or later:

GOLDAPPS11i
GOLDBASE11i
All installations Patches for HP-UX 11i (11.11):
  • PHCO_28123: cumulative SAM patch

  • PHKL_29198: Psets Enablement Patch; top(1)

  • PHNE_28476: Cumulative STREAMS Patch

  • PHNE_28923: LAN product cumulative patch

  • PHSS_28871: ld(1) and linker tools cumulative patch

  • PHSS_28880: HP aC++ -AA runtime libraries (aCC A.03.50)

All installations Patches for JDK on HP-UX 11i (11.11):
  • PHCO_26331: mountall cumulative patch, Dev IDs enabler

  • PHCO_29109: Pthread enhancement and fixes

  • PHKL_25468: eventport (/dev/poll) pseudo driver

  • PHKL_25842: Thread Abort

  • PHKL_25993: thread nostop for NFS, rlimit, Ufalloc fix

  • PHKL_25994: Thread NOSTOP, Psets Enablement, Ufalloc

  • PHKL_25995: ufalloc;VxFS3.5;SPP fragmentation

  • PHKL_26468: Shared synchronization performance support

  • PHKL_28489: copyin EFAULT, LDCD access type

Note: See the following Web site for information about additional patches that might be required by JDK 1.4.2:

http://www.hp.com/products1/unix/java/patches/index.html
Real Application Clusters Patches for HP-UX 11i (11.11):

PHSS_29096

Pro*C/C++, Oracle Call Interface, Oracle C++ Call Interface, Oracle XML Developer's Kit (XDK) Patches for HP-UX 11i (11.11):
  • PHSS_29484: ANSI C compiler B.11.11.08 cumulative patch

  • PHSS_29485: +O4/PBO Compiler B.11.11.08 cumulative patch

Oracle DCE Integration (part of Oracle Advanced Security) or Oracle Net protocol support for DCE Patches for HP-UX 11i (11.11):
  • PHSS_28386: HP DCE/9000 1.8 DCE Client IPv6 patch

  • PHSS_28387: HP DCE/9000 1.8 Server/DevTools cum. patch

  • PHSS_28388: HP DCE/9000 1.8 Integrated Login cum. patch

  • PHSS_29669: HP DCE/9000 1.8 DCE Client IPv6 patch

Oracle Messaging Gateway Corrective service diskettes (CSDs) for WebSphere MQ:
  • CSD05 or later for WebSphere MQ V5.3

To ensure that the system meets these requirements, follow these steps:

  1. To determine whether the HP-UX 11i Quality Pack is installed, enter the following command:

    # /usr/sbin/swlist -l bundle | grep GOLD
    
    

    If the Quality Pack is not installed, or if the date is before June 2003, download the latest Quality Pack from the following Web site and install it:

    http://www.software.hp.com/SUPPORT_PLUS/qpk.html
    
    
  2. To determine whether a patch is installed, enter a command similar to the following:

    # /usr/sbin/swlist -l patch | grep PHSS_28880
    
    

    Alternatively, to list all installed patches, enter the following command:

    # /usr/sbin/swlist -l patch | more
    
    

    If a required patch is not installed, download it from the following Web site and install it:

    http://itresourcecenter.hp.com
    
    

    If the Web site shows a more recent version of the patch, download and install that version.

  3. If you require a CSD for WebSphere MQ, see the following Web site for download and installation information:

    http://www.ibm.com/software/integration/mqfamily/support/summary/hpx.html
    

Create Required Symbolic Links


Note:

This task is required only if the Motif 2.1 Development Environment package (X11MotifDevKit.MOTIF21-PRG) is not installed.

To enable you to successfully relink Oracle products after installing this software, enter the following commands to create required X library symbolic links in the /usr/lib directory:

# cd /usr/lib
# ln -s libX11.3 libX11.sl
# ln -s libXIE.2 libXIE.sl
# ln -s libXext.3 libXext.sl
# ln -s libXhp11.3 libXhp11.sl
# ln -s libXi.3 libXi.sl
# ln -s libXm.4 libXm.sl
# ln -s libXp.2 libXp.sl
# ln -s libXt.3 libXt.sl
# ln -s libXtst.2 libXtst.sl

Create Required UNIX Groups and User

Depending on whether this is the first time Oracle software is being installed on this system and on the products that you are installing, you may need to create several UNIX groups and a UNIX user.

The following UNIX groups and user are required if you are installing Oracle Database:

The following UNIX group and user are required for all installations:

A single Oracle Inventory group is required for all installations of Oracle software on the system. After the first installation of Oracle software, you must use the same Oracle Inventory group for all subsequent Oracle software installations on that system. However, you can choose to create different Oracle software owner users, OSDBA groups, and OSOPER groups (other than oracle, dba, and oper) for separate installations. By using different groups for different installations, members of these different groups have DBA privileges only on the associated databases rather than on all databases on the system.


See Also:

The Oracle Database Administrator's Reference for UNIX Systems and the Oracle Database Administrator's Guide contain more information about the OSDBA and OSOPER groups and the SYSDBA and SYSOPER privileges.

The following sections describe how to create the required UNIX user and groups.


Note:

The following sections describe how to create local users and groups. As an alternative to creating local users and groups, you could create the appropriate users and groups in a directory service, for example, Network Information Services (NIS). For information about using directory services, contact your system administrator or see your operating system documentation.

Creating the Oracle Inventory Group

You must create the Oracle Inventory group if it does not already exist. The following subsections describe how to determine the Oracle Inventory group name, if it exists, and how to create it if necessary.

Determine Whether the Oracle Inventory Group Exists

When you install Oracle software on the system for the first time, the Installer creates the oraInst.loc file. This file identifies the name of the Oracle Inventory group and the path of the Oracle Inventory directory. To determine whether the Oracle Inventory group exists, enter the following command:

# more /var/opt/oracle/oraInst.loc

If the oraInst.loc file exists, the output from this command is similar to the following:

inventory_loc=/u01/app/oracle/oraInventory
inst_group=oinstall

The inst_group parameter shows the name of the Oracle Inventory group (oinstall).

Create the Oracle Inventory Group

If the oraInst.loc file does not exist, create the Oracle Inventory group, oinstall, by entering the following command:

# /usr/sbin/groupadd oinstall

Creating the OSDBA Group

You must create an OSDBA group in the following circumstances:

  • An OSDBA group does not exist, for example, if this is the first installation of Oracle Database software on the system

  • An OSBDA group exists, but you want to give a different group of UNIX users database administrative privileges in a new Oracle installation

If the OSDBA group does not exist or if you require a new OSDBA group, create it by entering the following command, using the group name dba unless a group with that name already exists:

# /usr/sbin/groupadd dba

Creating an OSOPER Group (Optional)

Create an OSOPER group only if you want to identify a group of UNIX users with a limited set of database administrative privileges (SYSOPER operator privileges). For most installations, it is sufficient to create only the OSDBA group. If you want to use an OSOPER group, you must create it in the following circumstances:

  • If an OSOPER group does not exist, for example, if this is the first installation of Oracle Database software on the system

  • If an OSOPER group exists, but you want to give a different group of UNIX users database operator privileges in a new Oracle installation

If you require an new OSOPER group, create it by entering the following command, using the group name oper unless a group with that name already exists:

# /usr/sbin/groupadd oper

Creating the Oracle Software Owner User

You must create an Oracle software owner user in the following circumstances:

  • If an Oracle software owner user does not exist, for example, if this is the first installation of Oracle software on the system

  • If an Oracle software owner user exists, but you want to use a different UNIX user, with different group membership, to give database administrative privileges to those groups in a new Oracle Database installation

Determine Whether an Existing Oracle Software Owner User Exists

To determine whether an Oracle software owner user named oracle exists, enter the following command:

# id oracle

If the oracle user exists, the output from this command is similar to the following:

uid=440(oracle) gid=200(oinstall) groups=201(dba),202(oper)

If the user exists, determine whether you want to use the existing user or create a new user. If you want to use the existing user, ensure that the user's primary group is the Oracle Inventory group and that it is a member of the appropriate OSDBA and OSOPER groups. See one of the following sections for more information:


Note:

If necessary, contact your system administrator before using or modifying an existing user.

Create a New Oracle Software Owner User

If the Oracle software owner user does not exist or if you require a new Oracle software owner user, create it as follows, using the user name oracle unless a user with that name already exists.

  1. To create the oracle user, enter a command similar to the following:

    # /usr/sbin/useradd -g oinstall -G dba[,oper] oracle
    
    

    In this command:

    • The -g option specifies the primary group, which must be the Oracle Inventory group, for example, oinstall

    • The -G option specifies the secondary groups, which must include the OSDBA group and if required, the OSOPER group, for example dba or dba,oper

  2. Set the password of the oracle user:

    # passwd oracle
    
    

See "Creating an Unprivileged User" to continue:

Modify an Existing Oracle Software Owner User

If the oracle user exists, but its primary group is not oinstall or it is not a member of the appropriate OSDBA or OSOPER groups, you can modify it by entering a command similar to the following, specifying the primary group using the -g option and any required secondary groups using the -G option:as follows:

# /usr/sbin/usermod -g oinstall -G dba[,oper] oracle

Creating an Unprivileged User

You must create an new unprivileged local user to own the extjob executable. You can use any user name for this user, but this document uses the name extjob. To create the extjob user, enter the following command on all cluster nodes:

# /usr/sbin/useradd extjob

Create Identical Users and Groups on Other Cluster Nodes


Note:

You must complete the following procedures only if you are using local users and groups. If you are using users and groups defined in a directory service such as NIS, they are already identical on each cluster node.

The Oracle software owner user and the Oracle Inventory, OSDBA, and OSOPER groups must exist and be identical on all cluster nodes. To create these identical users and groups, you must identify the user ID and group IDs assigned them on the node where you created them, then create the user and groups with the same name and ID on the other cluster nodes.

Identifying the User and Group IDs

To determine the user ID (UID) of the Oracle software owner user and the group IDs (GID) of the Oracle Inventory, OSDBA, and OSOPER groups, follow these steps:

  1. Enter the following command:

    # id oracle
    
    

    The output from this command is similar to the following:

    uid=440(oracle) gid=200(oinstall) groups=201(dba),202(oper)
    
    
  2. From the output, identify the UID for the oracle user and the GIDs for the groups to which it belongs.

Create the User and Groups on the Other Cluster Nodes

To create the user and groups on the other cluster nodes, repeat the following procedure on each node:

  1. Log in to the next cluster node as root.

  2. Enter commands similar to the following to create the oinstall and dba groups, and if required, the oper group. Use the -g option to specify the correct GID for each group.

    # /usr/sbin/groupadd -g 200 oinstall
    # /usr/sbin/groupadd -g 201 dba
    # /usr/sbin/groupadd -g 202 oper
    
    

    Note:

    If the group already exists, use the groupmod command to modify it if necessary. If you cannot use the same group ID for a particular group on this node, view the /etc/group file on all nodes to identify a group ID that is available on every node. You must then specify that ID for the group on all of the nodes.
  3. To create the oracle user, enter a command similar to the following:

    # /usr/sbin/useradd -u 200 -g oinstall -G dba[,oper] oracle
    
    

    In this command:

    • The -u option specifies the user ID, which must be the user ID that you identified in the previous subsection.

    • The -g option specifies the primary group, which must be the Oracle Inventory group, for example oinstall

    • The -G option specifies the secondary groups, which must include the OSDBA group and if required, the OSOPER group, for example dba or dba,oper


      Note:

      If the user already exists, use the usermod command to modify it if necessary. If you cannot use the same user ID for the oracle user on this node, view the /etc/passwd file on all nodes to identify a user ID that is available on every node. You must then specify that ID for the user on all of the nodes.
  4. Set the password of the oracle user:

    # passwd oracle
    
    

Set Up User Equivalence for rsh and rcp on All Cluster Nodes


Note:

This section describes how to set up user equivalence for rcp, which the Installer uses when copying Oracle software to the other cluster nodes. If you prefer, you can configure the Secure Shell (SSH) tool suite, so that the Installer uses scp instead of rcp. See the SSH documentation for information about setting up user equivalence for scp.

Before you install and use Oracle Real Application clusters, you must set up user equivalence on all cluster nodes, as follows:

  1. On all cluster nodes, edit the /etc/hosts.equiv file in any text editor and add entries similar to the following for all nodes in the cluster:

    node1-public oracle
    node1-public.domain oracle
    node2-public oracle
    node2-public.domain oracle
       .
       .
       .
    

    In this example:

    • noden_public is the host name associated with the primary public network interface

    • domain is the domain name of the system, for example mydomain.com


    Note:

    Alternatively, you can create a .rhosts file using the same format in the oracle user's home directory on all nodes in the cluster.
  2. To test user equivalence on all cluster nodes, including the first node, enter commands similar to the following from the node where you will run the Installer:

    # su oracle
    $ remsh nodename1 date
    $ remsh nodename2 date
        .
        .
        .
    $ exit
    
    

    Each command should return only the date setting on the specified node. If a node prompts for a password or displays any other text, then user equivalence is not set up correctly on that node. In this case, verify that the /etc/hosts.equiv (or ~oracle/.rhosts) file has the correct entries.

Grant Privileges to the OSDBA Group

To enhance the performance of the software, you must grant several system privileges to the OSDBA group by completing the following steps:

  1. Using any text editor, open the /etc/privgroup file, or create it if necessary.

  2. Add or edit the following line, which begins with the name of the OSDBA group, specifying the privileges MLOCK, RTPRIO, and RTSCHED, that you want to grant to this group every time the system reboots:

    dba MLOCK RTPRIO RTSCHED
    
    
  3. Save the file and exit from the text editor.

  4. Enter the following command to grant the privileges to the OSDBA group:

    # /usr/sbin/setprivgrp -f /etc/privgroup
    
    
  5. Enter the following command to verify that the privileges are set correctly:

    # /usr/sbin/getprivgrp dba
    
    
  6. Repeat this procedure on all of the other cluster nodes.

Configure Kernel Parameters


Notes:

The kernel parameter and shell limit values shown in this section are recommended values only. For production database systems, Oracle recommends that you tune these values to optimize the performance of the system. See your operating system documentation for more information about tuning kernel parameters.

You must set the kernel parameters on all cluster nodes.

Verify that the kernel parameters shown in the following table are set either to the formula shown, or to values greater than or equal to the recommended value shown. The procedure following the table describes how to verify and set the values.

Parameter Recommended Formula or Value
ksi_alloc_max (nproc*8)
max_thread_proc 256
maxdsiz 1073741824 (1 GB)
maxdsiz_64bit 2147483648 (2 GB)
maxssiz 134217728 (128 MB)
maxssiz_64bit 1073741824 (1 GB)
maxswapchunks 16384
maxuprc ((nproc*9)/10)
msgmap (2+msgmni)
msgmni 4096
msgseg 32767
msgtql 4096
ncsize (ninode+vx_ncsize)
nfile (15*nproc+2048)
nflocks 4096
ninode (8*nproc+2048)
nkthread (((nproc*7)/4)+16)
nproc 4096
semmap (semmni+2)
semmni 4096
semmns (semmni*2)
semmnu (nproc-4)
semvmx 32767
shmmax The size of physical memory (0X40000000) or 1073741824, whichever is greater.

Note: To avoid performance degradation, the value should be greater than or equal to the size of the SGA. If you are using Hyper Messaging protocol as the cluster interconnect, the value of this parameter must be less than 4 GB.

shmmni 512
shmseg 120
vps_ceiling 64

Note:

If the current value for any parameter is higher than the value listed in this table, do not change the value of that parameter.

To view the current value or formula specified for these kernel parameters, and to change them if necessary, follow these steps:

  1. Optionally, set the DISPLAY environment variable to specify the display of the local system:

    • Bourne, Bash, or Korn shell:

      $ DISPLAY=local_host:0.0 ; export DISPLAY
      
      
    • C shell:

      $ setenv DISPLAY local_host:0.0
      
      
  2. Start System Administration Manager (SAM):

    # /usr/sbin/sam
    
    
  3. Choose the Kernel Configuration area, then choose the Configurable Parameters area.

  4. Check the value or formula specified for each of these parameters and, if necessary, modify that value or formula.

    If necessary, see the SAM online help for more information about completing this step.

  5. Exit from SAM.

    If you modified the value specified for any static parameter, you must rebuild the kernel and reboot the system.

  6. If necessary, when the system restarts, log in and switch user to root.

  7. Complete this procedure on all other cluster nodes.

Identify Required Software Directories

You must identify or create four directories for the Oracle software, as follows:

The following subsections describe the requirements for these directories.

Oracle Base Directory

The Oracle base directory acts as a top-level directory for Oracle software installations. It is analogous to the C:\Oracle directory used for Oracle software installations on Windows systems. On UNIX systems, the Optimal Flexible Architecture (OFA) guidelines recommend that you use a path similar to the following for the Oracle base directory:

/mount_point/app/oracle_sw_owner

In this example:

  • mount_point is the mount point directory for the file system that will contain the Oracle software.

    The examples in this guide use /u01 for the mount point directory. However, you could choose another mount point directory, /oracle or /opt/oracle for example.

  • oracle_sw_owner is the UNIX user name of the Oracle software owner, for example oracle.

You can use the same Oracle base directory for more than one installation or you can create separate Oracle base directories for different installations. If different UNIX users install Oracle software on the same system, each user must create a separate Oracle base directory. The following example Oracle base directories could all exist on the same system:

/u01/app/oracle
/u01/app/orauser
/opt/oracle/app/oracle

The following sections describe how to identify existing Oracle base directories that might be suitable for your installation and how to create a new Oracle base directory if necessary.

Regardless of whether you create a new Oracle base directory or decide to use an existing one, you must set the ORACLE_BASE environment variable to specify the full path to this directory.


Note:

The Oracle base directory must be on a local file system.

Oracle Inventory Directory

The Oracle Inventory directory (oraInventory) stores an inventory of all software installed on the system. It is required by, and shared by, all Oracle software installations on a single system. The first time you install Oracle software on a system, the Installer prompts you to specify the path to this directory. If you are installing the software on a local file system, Oracle recommends that you choose the following path:

oracle_base/oraInventory

If the Oracle base directory is on a cluster file system, you must specify a path for the Oracle Inventory directory on a local file system, to enable all of the nodes to have separate inventories.

The Installer creates the directory that you specify and sets the correct owner, group, and permissions on it. You do not need to create it.


Note:

All Oracle software installations rely on this directory. Make sure that you back it up regularly.

Do not delete this directory unless you have completely removed all Oracle software from the system.

CRS Home Directory

The CRS home directory is the directory where you choose to install the software for Oracle Cluster Ready Services. You must install CRS in a separate home directory. When you run the Installer, it prompts you to specify the path to this directory, as well as a name that identifies it. The directory that you specify must be a subdirectory of the Oracle base directory. Oracle recommends that you specify a path similar to the following for the CRS home directory:

oracle_base/product/10.1.0/crs

The Installer creates the directory path that you specify under the Oracle base directory. It also sets the correct owner, group, and permissions on it. You do not need to create this directory.

Oracle Home Directory

The Oracle home directory is the directory where you choose to install the software for a particular Oracle product. You must install different Oracle products, or different releases of the same Oracle product, in separate Oracle home directories. When you run the Installer, it prompts you to specify the path to this directory, as well as a name that identifies it. The directory that you specify must be a subdirectory of the Oracle base directory. Oracle recommends that you specify a path similar to the following for the Oracle home directory:

oracle_base/product/10.1.0/db_1

The Installer creates the directory path that you specify under the Oracle base directory. It also sets the correct owner, group, and permissions on it. You do not need to create this directory.

Identify or Create an Oracle Base Directory

Before starting the installation, you must either identify an existing Oracle base directory or if required, create a new one. This section contains information about the following:


Note:

You can choose to create a new Oracle base directory, even if other Oracle base directories exist on the system.

Identifying an Existing Oracle Base Directory

Existing Oracle base directories might not have paths that comply with OFA guidelines. However, if you identify an existing Oracle Inventory directory or existing Oracle home directories, you can usually identify the Oracle base directories, as follows:

  • Identify an existing Oracle Inventory directory

    Enter the following command on all cluster nodes to view the contents of the oraInst.loc file:

    # more /var/opt/oracle/oraInst.loc
    
    

    If the oraInst.loc file exists, the output from this command is similar to the following:

    inventory_loc=/u01/app/oracle/oraInventory
    inst_group=oinstall
    
    

    The inventory_loc parameter identifies the Oracle Inventory directory (oraInventory) on that system. The parent directory of the oraInventory directory is typically an Oracle base directory. In the previous example, /u01/app/oracle is an Oracle base directory.

  • Identify existing Oracle home directories

    Enter the following command on all cluster nodes to view the contents of the oratab file:

    # more /etc/oratab
    
    

    If the oratab file exists, it contains lines similar to the following:

    *:/u03/app/oracle/product/10.1.0/db_1:N
    *:/opt/orauser/infra_904:N
    *:/oracle/9.2.0:N
    
    

    The directory paths specified on each line identify Oracle home directories. Directory paths that end with the user name of the Oracle software owner that you want to use are valid choices for an Oracle base directory. If you intend to use the oracle user to install the software, you could choose one of the following directories from the previous example:

    /u03/app/oracle
    /oracle
    

    Note:

    If possible, choose a directory path similar to the first (/u03/app/oracle). This path complies with the OFA guidelines.

Before deciding to use an existing Oracle base directory for this installation, make sure that it satisfies the following conditions:

  • It should not be on the same file system as the operating system.

  • It must have up to 4 GB of free disk space. Use the following command to determine the free disk space on the file system where the Oracle base directory is located:

    # bdf oracle_base_path

  • If an Oracle base directory does not exist on the system or if you want to create a new Oracle base directory, see the "Creating a New Oracle Base Directory" section.

  • If an Oracle base directory exists and you want to use it, see the "Configure Disk Storage for Oracle CRS and Database Files" section.

    When you are configuring the oracle user's environment later in this chapter, set the ORACLE_BASE environment variable to specify the directory you chose.

  • If an Oracle base directory does not exist on the system or if you want to create a new Oracle base directory, see the following section.

Creating a New Oracle Base Directory

Before you create a new Oracle base directory, you must identify an appropriate file system with at least 4 GB of free disk space.

To identify an appropriate file system, follow these steps:

  1. Use the bdf command to determine the free disk space on each mounted file system.

  2. From the display, identify a file system that has appropriate free space.


    Note:

    The file system must be a local file system.

    The path to the Oracle base directory must be the same on all nodes.

  3. Note the name of the mount point directory for the file system that you identified.

To create the Oracle base directory and specify the correct owner, group, and permissions for it, follow these steps:

  1. Enter commands similar to the following to create the recommended subdirectories in the mount point directory that you identified and set the appropriate owner, group, and permissions on them:

    # mkdir -p /mount_point/app/oracle_sw_owner
    # chown -R oracle:oinstall /mount_point/app
    # chmod -R 775 /mount_point/app
    
    

    If the mount point you identified is /u01 and oracle is the user name of the Oracle software owner, the recommended Oracle base directory path is as follows:

    /u01/app/oracle
    
    
  2. If necessary, repeat the commands listed in the previous step to create the same directory on the other cluster nodes.

  3. When you are configuring the oracle user's environment later in this chapter, set the ORACLE_BASE environment variable to specify this directory.

Configure Disk Storage for Oracle CRS and Database Files

This section describes the storage options available to you for storing Oracle Cluster Ready Services files, Oracle Database Files, and optionally, Oracle Database recovery files. After you choose the storage method that you want to use for each file type, see the following sections for information about configuring the required storage.


Note:

You do not have to use the same storage option for each type of file.

Choosing a Storage Option for Oracle CRS Files

Before you install Oracle Cluster Ready Services, you must choose the storage option that you want to use for the Oracle Cluster Registry (100 MB) and CRS voting disk (20 MB). You cannot use Automatic Storage Management to store these files, because they must be accessible before any Oracle instance starts.

Apart from ASM, you can use any of the storage options listed in the next section for the Oracle CRS files.


Note:

If you are not using HP Serviceguard, you must use raw partitions to store the Oracle CRS files. You cannot use shared raw logical volumes to store these files without HP Serviceguard.

Choosing a Storage Option for Oracle Database Files

If you want to create a database during the installation, you must choose one of the following storage options for the database files:

  • Automatic Storage Management

  • Shared logical volumes using HP Serviceguard Extensions for RAC and LVM

  • Raw partitions


Note:

For the most up-to-date information about supported storage options for RAC installations, see the Certify pages on the OracleMetaLink Web site:
http://metalink.oracle.com

Planning a Storage Option for Oracle Database Recovery Files

Additionally, if you want to enable automated backups during the installation, you must use Automatic Storage Management for recovery files (the flash recovery area).

Configuring Disk Storage

For information about how to configure disk storage before you start the installation, see one of the following sections depending on your choice:

Configure Disks for Automatic Storage Management

This section describes how to configure disks for use with ASM. Before you configure the disks, you must determine the number of disks and the amount of free disk space that you require. The following sections describe how to identify the requirements and configure the disks:


Note:

Although this section refers to disks, you can also use zero-padded files on a certified NAS storage device in an ASM disk group. See the appendices in Oracle Database Installation Guide for UNIX Systems for information about creating and configuring NAS-based files for use in an ASM disk group.

Identifying Storage Requirements for ASM

To identify the storage requirements for using ASM, you must determine how many devices and the amount of free disk space that you require. To complete this task, follow these steps:

  1. Determine whether you want to use ASM for Oracle database files (datafiles), recovery files, or both.


    Note:

    You do not have to use the same storage mechanism for database files and recovery files. One can use the file system, while the other uses ASM.

    If you enable automated backups during the installation, you can choose ASM as the storage mechanism for recovery files by specifying an ASM disk group for the flash recovery area. Depending how you choose to create a database during the installation, you have the following options:

    • If you select an installation method that runs DBCA in interactive mode, by choosing the Advanced database configuration option for example, you can decide whether you want to use the same ASM disk group for datafiles and recovery files, or you can choose to use different disk groups for each file type.

      The same choice is available to you if you use DBCA after the installation to create a database.

    • If you select an installation type that runs DBCA in non-interactive mode, you must use the same ASM disk group for database files and recovery files.

  2. Choose the ASM redundancy level that you want to use for the ASM disk group.

    The redundancy level that you choose for the ASM disk group determines how ASM mirrors files in the disk group and determines the number of disks and amount of disk space that you require, as follows:

    • External redundancy

      An external redundancy disk group requires a minimum of one disk device. The effective disk space in an external redundancy disk group is the sum of the disk space in all of its devices.

      Because ASM does not mirror data in an external redundancy disk group, Oracle recommends that you use only RAID or similar devices that provide their own data protection mechanisms as disk devices in this type of disk group.

    • Normal redundancy

      In a normal redundancy disk group, ASM uses two-way mirroring by default, to increase performance and reliability. A normal redundancy disk group requires a minimum of two disk devices (or two failure groups). The effective disk space in a normal redundancy disk group is half the sum of the disk space in all of its devices.

      For most installations, Oracle recommends that you use normal redundancy disk groups.

    • High redundancy

      In a high redundancy disk group, ASM uses three-way mirroring to increase performance and provide the highest level of reliability. A high redundancy disk group requires a minimum of three disk devices (or three failure groups). The effective disk space in a high redundancy disk group is one-third the sum of the disk space in all of its devices.

      While high redundancy disk groups do provide a high level of data protection, you must consider the higher cost of additional storage devices before deciding to use this redundancy level.

  3. Determine the total amount of disk space that you require for the datafiles and recovery files.

    Use the following table to determine the minimum number of disks and the minimum disk space requirements for the installation:

    Redundancy Level Minimum Number of Disks Datafiles Recovery FIles Both File Types
    External 1 1.15 GB 2.3 GB 3.45 GB
    Normal 2 2.3 GB 4.6 GB 6.9 GB
    High 3 3.45 GB 6.9 GB 10.35 GB

    You must also add additional disk space for the ASM metadata. You can use the following formula to calculate the additional disk space requirements (in MB):

    15 + (2 * number_of_disks) + (126 * number_of_ASM_instances)

    For example, for a four-node RAC installation, using three disks in a high redundancy disk group, you require an additional 525 MB of disk space:

    (15 + (2 * 3) +(126 * 4)) = 525

    If an existing ASM instance exists on the system, you can use an existing disk group to meet these storage requirements. If necessary, you can add disks to an existing disk group during the installation.

    The following section describes how to identify existing disk groups and determine the free disk space that they contain.

  4. Optionally identify failure groups for the ASM disk group devices.


    Note:

    You need to complete this step only if you intend to use an installation method that runs DBCA in interactive mode, for example, if you intend to choose the Custom installation type or the Advanced database configuration option. Other installation types do not enable you to specify failure groups.

    If you intend to use a normal or high redundancy disk group, you can further protect your database against hardware failure by associating a set of disk devices in a custom failure group. By default, each device comprises its own failure group. However, if two disk devices in a normal redundancy disk group are attached to the same SCSI controller, the disk group becomes unavailable if the controller fails. The controller in this example is a single point of failure.

    To avoid failures of this type, you could use two SCSI controllers, each with two disks, and define a failure group for the disks attached to each controller. This configuration would enable the disk group to tolerate the failure of one SCSI controller.


    Note:

    If you define custom failure groups, you must specify a minimum of two failure groups for normal redundancy disk groups and three failure groups for high redundancy disk groups.
  5. If you are sure that a suitable disk group does not exist on the system, install or identify appropriate disk devices to add to a new disk group. Use the following guidelines when identifying appropriate disk devices:

    • All of the devices in an ASM disk group should be the same size and have the same performance characteristics.

    • Do not specify more than one partition on a single physical disk as a disk group device. ASM expects each disk group device to be on a separate physical disk.

    • Although you can specify a logical volume as a device in an ASM disk group, Oracle does not recommend their use. Logical volume managers can hide the physical disk architecture, preventing ASM from optimizing I/O across the physical devices.

    For information about completing this task, see the "Configuring Disks for ASM" section.

Using an Existing ASM Disk Group

If you want to use ASM as the storage option for either database or recovery files, and an existing ASM disk group exists, you have the following choices, depending on the installation method that you select:

  • If you select an installation method that runs DBCA in interactive mode, by choosing the Advanced database configuration option for example, you can decide whether you want to create a new disk group or use an existing one.

    The same choice is available to you if you use DBCA after the installation to create a database.

  • If you select an installation type that runs DBCA in non-interactive mode, you must choose an existing disk group for the new database; you cannot create a new disk group. However, you can add disk devices to an existing disk group if it has insufficient free space for your requirements.


Note:

The ASM instance that manages the existing disk group can be running in a different Oracle home directory.

To determine whether an existing ASM disk group exists, or to determine whether there is sufficient disk space in a disk group, you can use Oracle Enterprise Manager Database Grid Control or Database Control. Alternatively, you can use the following procedure:

  1. View the contents of the oratab file to determine whether an ASM instance is configured on the system:

    # more /etc/oratab
    
    

    If an ASM instance is configured on the system, the oratab file should contain a line similar to the following:

    +ASM:oracle_home_path:N
    
    

    In this example, +ASM is the system identifier (SID) of the ASM instance and oracle_home_path is the Oracle home directory where it is installed. By convention, the SID for an ASM instance begins with a plus sign.

  2. Set the ORACLE_SID and ORACLE_HOME environment variables to specify the appropriate values for the ASM instance that you want to use.

  3. Connect to the ASM instance as the SYS user with SYSDBA privilege and start the instance if necessary:

    # $ORACLE_HOME/bin/sqlplus "SYS/SYS_password as SYSDBA"
    SQL> STARTUP
    
    
  4. Enter the following command to view the existing disk groups, their redundancy level, and the amount of free disk space in each one:

    SQL> SELECT NAME,TYPE,TOTAL_MB,FREE_MB FROM V$ASM_DISKGROUP;
    
    
  5. From the output, identify a disk group with the appropriate redundancy level and note the free space that it contains.

  6. If necessary, install or identify the additional disk devices required to meet the storage requirements listed in the previous section.


    Note:

    If you are adding devices to an existing disk group, Oracle recommends that you use devices that have the same size and performance characteristics as the existing devices in that disk group.

Configuring Disks for ASM

To configure disks for use with ASM, follow these steps:

  1. If necessary, install the shared disks that you intend to use for the ASM disk group.

  2. To make sure that the disks are available, enter the following command:

    # /usr/sbin/ioscan -fun -C disk
    
    

    The output from this command is similar to the following:

    Class  I  H/W Path    Driver S/W State   H/W Type     Description
    ==========================================================================
    disk    0  0/0/1/0.6.0 sdisk  CLAIMED     DEVICE       HP   DVD-ROM 6x/32x
                           /dev/dsk/c0t6d0   /dev/rdsk/c0t6d0
    disk    1  0/0/1/1.2.0 sdisk  CLAIMED     DEVICE      SEAGATE ST39103LC
                           /dev/dsk/c1t2d0   /dev/rdsk/c1t2d0
    
    

    This command displays information about each disk attached to the system, including the block device name (/dev/dsk/cxtydz) and the character raw device name (/dev/rdsk/cxtydz).

    If this command does not display device name information, enter the following command to install the special device files:

    # /usr/sbin/insf -e
    
    
  3. For each disk that you want to add to a disk group, enter the following command to verify that it is not already part of an LVM volume group:

    # /sbin/pvdisplay /dev/dsk/cxtydz
    
    

    If this command displays volume group information, the disk is already part of a volume group. The disks that you choose must not be part of an LVM volume group.


    Note:

    If you are using different volume management software, for example VERITAS Volume Manager, see the appropriate documentation for information about verifying that a disk is not in use.
  4. Enter commands similar to the following to change the owner, group, and permissions on the character raw device file for each disk that you want to add to a disk group:

    # chown oracle:dba /dev/rdsk/cxtydz
    # chmod 660 /dev/rdsk/cxtydz
    
    

    Note:

    If you are using a multi-pathing disk driver with ASM, ensure that you set the permissions only on the correct logical device name for the disk.

Configure Raw Logical Volumes

See the following sections for information about configuring raw logical volumes:


Note:

If you are using ASM for database file storage, you need only create raw devices for the Oracle CRS files.

The following subsections describe how to create logical volumes using the command line. If you prefer, you can use SAM to complete the same tasks. See the HP-UX documentation for more information about using SAM.

Review Important Information

This section describes how to configure shared raw logical volumes for Oracle CRS and database file storage. The procedures in this section describe how to create a new shared volume group that contains the logical volumes required for both types of files.

To use shared raw logical volumes, HP Serviceguard Extensions for RAC must be installed on all cluster nodes. If HP Serviceguard Extensions for RAC is not installed, you can use raw partitions on a shared disk to store the Oracle CRS files. However, Oracle recommends that you use this method only for the CRS files and use an alternative method such as ASM for database file storage.

Before you continue, review the following guidelines which contain important information about using shared logical volumes with this release of Oracle Real Application Clusters:

  • You must use shared volume groups for Oracle CRS and database files.

  • If you specify raw logical volumes for the Oracle Cluster Registry and Oracle CRS voting disk during the Oracle CRS installation, the installation process configures the Oracle CRS startup scripts to activate the volume group that contains these logical volumes when Oracle CRS starts.


    Note:

    You do not need to create a Serviceguard package to activate the volume group that contains the Oracle CRS files.
  • The Oracle CRS files require less than 200 MB of disk space. To make efficient use of the disk space in a volume group, Oracle recommends that you use the same shared volume group for the logical volumes for both the CRS files and the database files.

  • If you are upgrading an existing Oracle9i release 2 RAC installation that uses raw logical volumes, you can use the existing SRVM configuration repository logical volume for the OCR and create a new logical volume in the same volume group for the Oracle CRS voting disk. However, before you install Oracle CRS, you must remove this volume group from any Serviceguard package that currently activates it.


    See Also:

    For information about removing a volume group from a Serviceguard package, see the HP Serviceguard or HP Serviceguard Extensions for RAC documentation.

    Note:

    If you are upgrading a database, you must also create a new logical volume for the SYSAUX tablespace. See the "Create Raw Logical Volumes in the New Volume Group" section for more information about the requirements for the Oracle CRS voting disk and SYSAUX logical volumes.
  • You must use either your own startup script or a Serviceguard package to activate new or existing volume groups that contain only database files (not Oracle CRS files).


    See Also:

    For information about creating a Serviceguard package to activate a shared volume group for RAC, see the HP Serviceguard documentation.
  • All shared volume groups that you intend to use for Oracle CRS or database files must be activated in shared mode before you start the installation.

  • All shared volume groups that you are using for RAC, including the volume group that contains the Oracle CRS files, must be specified in the cluster configuration file using the parameter OPS_VOLUME_GROUP.


    Note:

    If you create a new shared volume group for RAC on an existing HP Serviceguard cluster, you must reconfigure and restart the cluster before installing Oracle CRS. See the HP Serviceguard documentation for information about configuring the cluster and specifying shared volume groups.
  • The procedures in this section describe how to create basic volumes groups and volumes. If you want to configure more complex volumes, using mirroring for example, use this section in conjunction with the HP Serviceguard documentation.

Disable Operating System Activation of Shared Volume Groups

To prevent the operating system from activating shared volume groups when it boots, you must edit the /etc/lvmrc file, as follows:

  1. Create a backup copy of the /etc/lvmrc file:

    # cp /etc/lvmrc /etc/lvmrc_orig
    
    
  2. Open the /etc/lvmrc file in any text editor and search for the AUTO_VG_ACTIVATE flag.

  3. If necessary, change the value of the AUTO_VG_ACTIVATE flag to 0, to disable automatic volume group activation, as follows:

    AUTO_VG_ACTIVATE=0
    
    
  4. Search for the custom_vg_activation function in the /etc/lvmrc file.

  5. Add vgchange commands to the function, as shown in the following example, to automatically activate existing local volume groups:

    custom_vg_activation()
    {
            # e.g. /sbin/vgchange -a y -s
            #      parallel_vg_sync "/dev/vg00 /dev/vg01"
            #      parallel_vg_sync "/dev/vg02 /dev/vg03"
    
            /sbin/vgchange -a y vg00
            /sbin/vgchange -a y vg01
            /sbin/vgchange -a y vg02
    
            return 0
    }
    
    

    In this example, vg00, vg01, and vg02 are the volume groups that you want to activate automatically when the system reboots.


    Note:

    You must not specify any shared volume groups in this file. The shared volume group that you use for Oracle CRS files will be activated by the CRS startup scripts.

Create a Volume Group

To create a volume group, follow these steps:

  1. If necessary, install the shared disks that you intend to use for the database.

  2. To make sure that the disks are available, enter the following command:

    # /sbin/ioscan -fun -C disk
    
    

    The output from this command is similar to the following:

    Class  I  H/W Path    Driver S/W State   H/W Type     Description
    ==========================================================================
    disk    0  0/0/1/0.6.0 sdisk  CLAIMED     DEVICE       HP   DVD-ROM 6x/32x
                           /dev/dsk/c0t6d0   /dev/rdsk/c0t6d0
    disk    1  0/0/1/1.2.0 sdisk  CLAIMED     DEVICE      SEAGATE ST39103LC
                           /dev/dsk/c1t2d0   /dev/rdsk/c1t2d0
    disk    2  0/0/2/0.2.0 sdisk  CLAIMED     DEVICE       SEAGATE ST118202LC
                           /dev/dsk/c2t2d0   /dev/rdsk/c2t2d0
    
    

    This command displays information about each disk attached to the system, including the block device name (/dev/dsk/cxtydz) and the character raw device name (/dev/rdsk/cxtydz).

    If this command does not display device name information for a disk, enter the following command to install the special device files:

    # /usr/sbin/insf -e
    
    
  3. For each disk that you want to add to the volume group, enter the following command to verify that it is not already part of a volume group:

    # /sbin/pvdisplay /dev/dsk/cxtydz
    
    

    If this command displays volume group information, the disk is already part of a volume group.

  4. For each disk that you want to add to the volume group, enter a command similar to the following:

    # /sbin/pvcreate /dev/rdsk/cxtydz
    
    
  5. To create a directory for the volume group in the /dev directory, enter a command similar to the following, where vg_name is the name you want to use for the volume group:

    # mkdir /dev/vg_name
    
    
  6. To identify used device minor numbers, enter the following command on each node of the cluster:

    # ls -la /dev/*/group
    
    

    This command displays information about the device numbers used by all configured volume groups, similar to the following:

    crw-r-----   1 root    sys        64 0x000000 Mar  4  2002 /dev/vg00/group
    crw-r--r--   1 root    sys        64 0x010000 Mar  4  2002 /dev/vg01/group
    
    

    In this example, 64 is the major number used by all volume group devices and 0x000000 and 0x010000 are the minor numbers used by volume groups vg00 and vg01 respectively. Minor numbers have the format 0xnn0000, where nn is a number in the range 00 to the value of the maxvgs kernel parameter minus 1. The default value for the maxvgs parameter is 10, so the default range is 00 to 09.

  7. Identify an appropriate minor number that is unused on all nodes in the cluster.

  8. To create the volume group and activate it, enter commands similar to the following:

    # /sbin/mknod /dev/vg_name/group c 64 0xnn0000
    # /sbin/vgcreate /dev/vg_name /dev/dsk/cxtydz . . .
    # /sbin/vgchange -a y vg_name
    
    

    In this example:

    • vg_name is the name you want to give the volume group

    • 0xnn0000 is a minor number that is unused on all nodes in the cluster.

    • /dev/dsk/cxtydz... is a list of one or more block device names for the disks that you want to add to the volume group.

Create Raw Logical Volumes in the New Volume Group

To create the required raw logical volumes in the new volume group, follow these steps:

  1. Choose a name for the database that you want to create.

    The name that you choose must start with a letter and have no more than four characters, for example, orcl.

  2. From the following table, identify the logical volumes that you must create:

    Number Partition Size (MB) Purpose and Sample Logical Volume Name
    Raw Logical Volumes for Oracle Database
    1 500 SYSTEM tablespace:
    dbname_system_raw_500m
    
    1 300 + (Number of instances * 250) SYSAUX tablespace:
    dbname_sysaux_raw_800m
    
    Number of instances 500 UNDOTBSn tablespace (One tablespace for each instance, where n is the number of the instance):
    dbname_undotbsn_raw_500m
    
    1 160 EXAMPLE tablespace:
    dbname_example_raw_160m
    
    1 120 USERS tablespace:
    dbname_users_raw_120m
    
    2 * number of instances 120 Two online redo log files for each instance (where n is the number of the instance and m is the log number, 1 or 2):
    dbname_redon_m_raw_120m
    
    2 110 First and second control files:
    dbname_control[1|2]_raw_110m
    
    1 250 TEMP tablespace:
    dbname_temp_raw_250m
    
    1 5 Server parameter file (SPFILE):
    dbname_spfile_raw_5m
    
    1 5 Password file:
    dbname_pwdfile_raw_5m
    
    Raw Logical Volumes for Oracle Cluster Ready Services (CRS)
    1 100 Oracle Cluster Registry:
    ora_ocr_raw_100m
    

    Note: You need to create this raw logical volume only once on the cluster. If you create more than one database on the cluster, they all share the same Oracle cluster registry.

    If you are upgrading from Oracle9i Release 2, you can continue to use the raw device that you used for the SRVM configuration repository instead of creating this new logical volume.

    1 20 Oracle CRS voting disk:
    ora_vote_raw_20m
    

    Note: You need to create this raw logical volume only once on the cluster. If you create more than one database on the cluster, they all share the same Oracle CRS voting disk.

  3. To create each required logical volume, enter a command similar to the following:

    # /sbin/lvcreate -n LVname -L size /dev/VGname
    
    

    In this example:

    • LVname is the name of the logical volume that you want to create

      Oracle recommends that you use the sample names shown in the previous table for the logical volumes. Substitute the dbname variable in the sample logical volume name with the name you choose for the database in step 1.

    • VGname is the name of the volume group where you want to create the logical volume

    • size is the size of the logical volume in megabytes

    The following example shows a sample command used to create an 800 MB logical volume in the oracle_vg volume group for the SYSAUX tablespace of a database named test:

    # /sbin/lvcreate -n test_sysaux_800m -L 800 /dev/oracle_vg
    
    
  4. Change the owner, group, and permissions on the character device files associated with the logical volumes that you created, as follows:

    # chown oracle:dba /dev/vg_name/r*
    # chmod 755 /dev/vg_name
    # chmod 660 /dev/vg_name/r*
    
    
  5. Change the owner and group on the character device file associated with the logical volume for the Oracle Cluster Registry, as follows:

    # chown root:dba /dev/vg_name/rora_ocr_raw_100m
    
    

Export the Volume Group and Import It on the Other Cluster Nodes

To export the volume group and import it on the other cluster nodes, follow these steps:

  1. Deactivate the volume group:

    # /sbin/vgchange -a n vg_name
    
    
  2. To export the description of the volume group and its associated logical volumes to a map file, enter a command similar to the following:

    # /sbin/vgexport -v -s -p -m /tmp/vg_name.map /dev/vg_name
    
    
  3. Enter commands similar to the following to copy the map file to the other cluster nodes:

    # rcp /tmp/vg_name.map nodename:/tmp/vg_name.map
    
    
  4. Enter commands similar to the following on the other cluster nodes to import the volume group that you created on the first node:

    # mkdir /dev/vg_name
    # /sbin/mknod /dev/vg_name/group c 64 0xnn0000
    # /sbin/vgimport -v -s -m /tmp/vg_name.map /dev/vg_name
    
    
  5. Enter commands similar to the following on the other cluster nodes to change the owner, group, and permissions on the character device files associated with the logical volumes that you created:

    # chown oracle:dba /dev/vg_name/r*
    # chmod 755 /dev/vg_name
    # chmod 660 /dev/vg_name/r*
    
    
  6. Change the owner and group on the character device file associated with the logical volume for the Oracle Cluster Registry, as follows:

    # chown root:dba /dev/vg_name/rora_ocr_raw_100m
    
    

Activate the Volume Group in Shared Mode on All Cluster Nodes

To activate the volume group in shared mode on all cluster nodes, enter the following command on each node:

# /sbin/vgchange -a s vg_name


Note:

After you install Oracle CRS, the CRS startup scripts use this command to activate the volume group automatically when the node boots.

Create the DBCA Raw Device Mapping File


Note:

You must complete this procedure only if you are using raw logical volumes for database files. You do not specify the raw logical volumes (or partitions) for the Oracle CRS files in the DBCA raw device mapping file.

To enable Database Configuration Assistant (DBCA) to identify the appropriate raw device for each database file, you must create a raw device mapping file, as follows:

  1. Set the ORACLE_BASE environment variable to specify the Oracle base directory that you identified or created previously:

    • Bourne, Bash or Korn shell:

      $ ORACLE_BASE=/u01/app/oracle ; export ORACLE_BASE
      
      
    • C shell:

      $ setenv ORACLE_BASE /u01/app/oracle
      
      
  2. Create a database file subdirectory under the Oracle base directory and set the appropriate owner, group, and permissions on it:

    # mkdir -p $ORACLE_BASE/oradata/dbname
    # chown -R oracle:oinstall $ORACLE_BASE/oradata
    # chmod -R 775 $ORACLE_BASE/oradata
    
    

    In this example, dbname is the name of the database that you chose previously.

  3. Change directory to the $ORACLE_BASE/oradata/dbname directory.

  4. Enter a command similar to the following to create a text file that you can use to create the raw device mapping file:

    # find /dev/vg_name -user oracle -name 'r*' -print  dbname_raw.conf
    
    
  5. Edit the dbname_raw.conf file in any text editor to create a file similar to the following:


    Note:

    The following example shows a sample mapping file for a two-instance RAC cluster.
    system=/dev/vg_name/rdbname_system_raw_500m
    sysaux=/dev/vg_name/rdbname_sysaux_raw_800m
    example=/dev/vg_name/rdbname_example_raw_160m
    users=/dev/vg_name/rdbname_users_raw_120m
    temp=/dev/vg_name/rdbname_temp_raw_250m
    undotbs1=/dev/vg_name/rdbname_undotbs1_raw_500m
    undotbs2=/dev/vg_name/rdbname_undotbs2_raw_500m
    redo1_1=/dev/vg_name/rdbname_redo1_1_raw_120m
    redo1_2=/dev/vg_name/rdbname_redo1_2_raw_120m
    redo2_1=/dev/vg_name/rdbname_redo2_1_raw_120m
    redo2_2=/dev/vg_name/rdbname_redo2_2_raw_120m
    control1=/dev/vg_name/rdbname_control1_raw_110m
    control2=/dev/vg_name/rdbname_control2_raw_110m
    spfile=/dev/vg_name/rdbname_spfile_raw_5m
    pwdfile=/dev/vg_name/rdbname_pwdfile_raw_5m
    
    

    In this example:

    • vg_name is the name of the volume group

    • dbname is the name of the database

    Use the following guidelines when creating or editing this file:

    • Each line in the file must have the following format:

      database_object_identifier=logical_volume
      
      

      The logical volume names suggested in this manual include the database object identifier that you must use in this mapping file. For example, in the following logical volume name, redo1_1 is the database object identifier:

      /dev/oracle_vg/rac_redo1_1_raw_120m
      
      
    • For a single instance database, the file must specify one automatic undo tablespace data file (undotbs1), and at least two redo log files (redo1_1, redo1_2).

    • For a RAC database, the file must specify one automatic undo tablespace datafile (undotbsn) and two redo log files (redon_1, redon_2) for each instance.

    • Specify at least two control files (control1, control2).

    • To use manual instead of automatic undo management, specify a single RBS tablespace datafile (rbs) instead of the automatic undo management tablespaces.

  6. Save the file and note the file name that you specified.

  7. When you are configuring the oracle user's environment later in this chapter, set the DBCA_RAW_CONFIG environment variable to specify the full path to this file.

Verify the Cluster Software Configuration


Note:

The cluster software mentioned in the following sections is not required to install Oracle Real Application Clusters. However, if this software is installed, Oracle Cluster Ready Services (CRS) can integrate with it.

To verify that the HP Serviceguard software is running:

  1. If necessary, configure (or reconfigure) the cluster.

  2. Verify that HP Serviceguard is configured to start automatically at runlevel 3.

  3. To verify that HP Serviceguard is running correctly, enter the following command:

    # /usr/sbin/cmviewcl -v
    
    

    Verify that the cluster status is up, and that all nodes are up and running.

  4. If the cluster is not running, enter the following command to start it:

    # /usr/sbin/cmruncl
    
    
  5. If any nodes are not running, enter a command similar to the following to start the cluster daemon on those nodes:

    # /usr/sbin/cmrunnode nodename1 nodename2...
    
    

If you intend to use Hyper Messaging protocol as the cluster interconnect, enter the following command to verify that the HyperFabric software is configured properly:

# /opt/clic/bin/clic_stat -d NET

In the output from this command, verify that all of the cluster nodes are listed in the Node Table Entries section. If a node is missing, see the HyperFabric documentation for more information about configuring the software.

Stop Existing Oracle Processes


Caution:

If you are installing additional Oracle Database 10g products in an existing Oracle home, stop all processes running in the Oracle home. You must complete this task to enable the Installer to relink certain executables and libraries.

If you choose to create a database during the installation, most installation types configure and start a default Oracle Net listener using TCP/IP port 1521 and the IPC key value EXTPROC. However, if an existing Oracle Net listener process is using the same port or key value, the Installer can only configure the new listener; it cannot start it. To ensure that the new listener process starts during the installation, you must shut down any existing listeners before starting the Installer.

To determine whether an existing listener process is running and to shut it down if necessary, follow these steps:

  1. Switch user to oracle:

    # su - oracle
    
    
  2. Enter the following command to determine whether a listener process is running and to identify its name and the Oracle home directory in which it is installed:

    $ ps -ef | grep tnslsnr
    
    

    This command displays information about the Oracle Net listeners running on the system:

    ... oracle_home1/bin/tnslsnr LISTENER -inherit
    
    

    In this example, oracle_home1 is the Oracle home directory where the listener is installed and LISTENER is the listener name.

  3. Set the ORACLE_HOME environment variable to specify the appropriate Oracle home directory for the listener:

    • Bourne, Bash, or Korn shell:

      $ ORACLE_HOME=oracle_home1
      $ export ORACLE_HOME
      
      
    • C or tcsh shell:

      $ setenv ORACLE_HOME oracle_home1
      
      
  4. Enter the following command to identify the TCP/IP port number and IPC key value that the listener is using:

    $ $ORACLE_HOME/bin/lsnrctl status listenername
    

    Note:

    If the listener uses the default name LISTENER, you do not have to specify the listener name in this command.
  5. Enter a command similar to the following to stop the listener process:

    $ $ORACLE_HOME/bin/lsnrctl stop listenername
    
    
  6. Repeat this procedure to stop all listeners running on this system and on all other cluster nodes.

Configure the oracle User's Environment

You run the Installer from the oracle account. However, before you start the Installer you must configure the environment of the oracle user. To configure the environment, you must:

To set the oracle user's environment, follow these steps:

  1. Start a new terminal session, for example, an X terminal (xterm).

  2. Enter the following command to ensure that X Window applications can display on this system:

    $ xhost +
    
    
  3. If you are not already logged in to the system where you want to install the software, log in to that system as the oracle user.

  4. If you are not logged in as the oracle user, switch user to oracle:

    $ su - oracle
    
    
  5. To determine the default shell for the oracle user, enter the following command:

    $ echo $SHELL
    
    
  6. Open the oracle user's shell startup file in any text editor:

    • Bourne shell (sh), Bash shell (bash), or Korn shell (ksh):

      $ vi .profile
      
      
    • C shell (csh or tcsh):

      % vi .login
      
      
  7. Enter or edit the following line, specifying a value of 022 for the default file creation mask:

    umask 022
    
    
  8. If the ORACLE_SID, ORACLE_HOME, or ORACLE_BASE environment variables are set in the file, remove the appropriate lines from the file.

  9. Save the file and exit from the editor.

  10. To run shell startup script, enter one of the following command:

    • Bourne, Bash, or Korn shell:

      $ . ./.profile
      
      
    • C shell:

      % source ./.login
      
      
  11. If you are not installing the software on the local system, enter a command similar to the following to direct X applications to display on the local system:

    • Bourne, Bash, or Korn shell:

      $ DISPLAY=local_host:0.0 ; export DISPLAY
      
      
    • C shell:

      $ setenv DISPLAY local_host:0.0
      
      

    In this example, local_host is the host name or IP address of the system you want to use to display the Installer (your workstation or PC).

  12. If you determined that the /tmp directory has less than 200 MB of free disk space, identify a file system with at least 200 MB of free space and set the TEMP and TMPDIR environment variables to specify a temporary directory on this file system:

    1. Use the bdf command to identify a suitable file system with sufficient free space.

    2. If necessary, enter commands similar to the following to create a temporary directory on the file system you identified, and set the appropriate permissions on the directory:

      $ su - root
      # mkdir /mount_point/tmp
      # chmod a+wr /mount_point/tmp
      # exit
      
      
    3. Enter commands similar to the following to set the TEMP and TMPDIR environment variables:

      • Bourne, Bash, or Korn shell:

        $ TEMP=/mount_point/tmp
        $ TMPDIR=/mount_point/tmp
        $ export TEMP TMPDIR
        
        
      • C shell:

        $ setenv TEMP /mount_point/tmp
        $ setenv TMPDIR /mount_point/tmp
        
        
  13. Enter commands similar to the following to set the ORACLE_BASE environment variable:

    • Bourne, Bash, or Korn shell:

      $ ORACLE_BASE=/u01/app/oracle
      $ export ORACLE_BASE
      
      
    • C shell:

      % setenv ORACLE_BASE /u01/app/oracle
      
      

    In these examples, /u01/app/oracle is the Oracle base directory that you created earlier.

  14. If necessary, set the following environment variables. If you must specify several values for an environment variable, for example PATH, separate the values with a colon (:).

    Environment Variable Required By Sample Setting and Description
    DBCA_RAW_CONFIG Oracle Database installations that use raw devices for database storage. $ORACLE_BASE/oradata/dbname/dbname_raw.conf

    Specifies the location of the raw device mapping file

    COBDIR Pro*COBOL /opt/lib/cobol

    Specifies the directory where COBOL is installed on the system.

    Note: To install Pro*COBOL, you must choose a custom software installation.

    PATH Pro*COBOL $COBDIR/bin:$PATH

    Specifies the directory where the COBOL compiler executable is located.

  15. Enter the following command to ensure that the ORACLE_HOME and TNS_ADMIN environment variable are not set:

    $ unset ORACLE_HOME
    $ unset TNS_ADMIN
    
    

    Note:

    If the ORACLE_HOME environment variable is set, the Installer uses the value it specifies as the default path for the Oracle home directory. However, if you set the ORACLE_BASE environment variable, Oracle recommends that you unset this environment variable and choose the default path suggested by the Installer.
  16. To verify that the environment has been set correctly, enter the following commands:

    $ umask
    $ env | more
    
    

    Verify that the umask command displays a value of 22, 022, or 0022 and the environment variables that you set in this section have the correct values.