4 Configuring Oracle Clusterware Storage

4.1.1 Overview of Storage Options

Use the information in this overview to help you select your storage option.

https://metalink.oracle.com

4.1.2 Checking for Available Shared Storage with CVU

To check for all shared file systems available across all nodes on the cluster on a supported shared file system, use the following command:

/mountpoint/clusterware/cluvfy/runcluvfy.sh comp ssa -n node_list

If you want to check the shared accessibility of a specific shared storage type to specific nodes in your cluster, then use the following command syntax:

/mountpoint/clusterware/cluvfy/runcluvfy.sh comp ssa -n node_list -s storageID_list

In the preceding syntax examples, the variable mountpoint is the mountpoint path of the installation media, the variable node_list is the list of nodes you want to check, separated by commas, and the variable storageID_list is the list of storage device IDs for the storage devices managed by the file system type that you want to check.

For example, if you want to check the shared accessibility from node1 and node2 of storage devices /dev/c0t0d0s2 and /dev/c0t0d0s3, and your mountpoint is /dev/dvdrom/, then enter the following command:

/dev/dvdrom/clusterware/cluvfy/runcluvfy.sh comp ssa -n node1,node2 -s\ /dev/c0t0d0s2,/dev/c0t0d0s3

If you do not specify specific storage device IDs in the command, then the command searches for all available storage devices connected to the nodes on the list

4.2.1 Requirements for Using a File System for Oracle Clusterware Files

To use a file system for Oracle Clusterware files, the file system must comply with the following requirements:

• To use an NFS file system, it must be on a certified NAS device.

  Note:

  If you are using a shared file system on a NAS device to store a shared Oracle home directory for Oracle Clusterware or RAC, then you must use the same NAS device for Oracle Clusterware file storage.

• If you choose to place your Oracle Cluster Registry (OCR) files on a shared file system, then one of the following must be true:

  • The disks used for the file system are on a highly available storage device, (for example, a RAID device that implements file redundancy)

  • At least two file systems are mounted, and use the features of Oracle Database 11g Release 1 (11.1) to provide redundancy for the OCR.

  In addition, if you put the OCR and voting disk files on a shared file system, then that shared files system must be a shared QFS file system, and not a globally mounted UFS or VxFS file system.

• If you intend to use a shared file system to store database files, then use at least two independent file systems, with the database files on one file system, and the recovery files on a different file system.

• The oracle user must have write permissions to create the files in the path that you specify.

Use Table 4-1 to determine the partition size for shared file systems.

In Table 4-1, the total required volume size is cumulative. For example, to store all files on the shared file system with normal redundancy, you should have at least 1.3 GB of storage available over a minimum of three volumes (two separate volume locations for the OCR and OCR mirror, and one voting disk on each volume).

[root@node1]$ fdisk -s /dev/sdb1
281106

4.2.2 Checking UDP Parameter Settings

The User Data Protocol (UDP) parameter settings define the amount of send and receive buffer space for sending and receiving datagrams over an IP network. These settings affect cluster interconnect transmissions. If the buffers set by these parameters are too small, then incoming UDP datagrams can be dropped due to insufficient space, which requires send-side retransmission. This can result in poor cluster performance.

On Solaris, the UDP parameters are udp_recv_hiwat and udp_xmit_hiwat. On Solaris 10 the default values for these parameters are 57344 bytes. Oracle recommends that you set these parameters to at least 65536 bytes.

To check current settings for udp_recv_hiwat and udp_xmit_hiwat, enter the following commands:

# ndd /dev/udp udp_xmit_hiwat
# ndd /dev/udp udp_recv_hiwat

On Solaris 10, to set the values of these parameters to 65536 bytes in current memory, enter the following commands:

# ndd -set /dev/udp udp_xmit_hiwat 65536# ndd -set /dev/udp udp_recv_hiwat 65536

On Solaris 9, to set the values of these parameters to 65536 bytes on system restarts, open the /etc/system file, and enter the following lines:

set udp:udp_xmit_hiwat=65536
set udp:udp_recv_hiwat=65536

On Solaris 10, to set the UDP values for when the system restarts, the ndd commands have to be included in a system startup script. For example, The following script in /etc/rc2.d/S99ndd sets the parameters:

ndd -set /dev/udp udp_xmit_hiwat 65536 
ndd -set /dev/udp udp_recv_hiwat 65536

4.2.3 Checking NFS Buffer Size Parameters

If you are using NFS, then you must set the values for the NFS buffer size parameters rsize and wsize to at least 16384. Oracle recommends that you use the value 32768. Update the /etc/vfstab file on each node with an entry similar to the following:

nfs_server:/vol/DATA/oradata  /home/oracle/netapp nfs -yes
rw,hard,nointr,rsize=32768,wsize=32768,tcp,noac,vers=3

If you use NFS mounts, then Oracle recommends that you use the option forcedirectio to force direct I/O for better performance. However, if you add forcedirectio to the mount option, then the same mount point cannot be used for Oracle software binaries, executables, shared libraries, and objects. You can only use the forcedirectio option for Oracle data files, the OCR, and voting disks. For these mount points, enter the following line:

nfs_server:/vol/DATA/oradata  /home/oracle/netapp nfs -yes
rw,hard,nointr,rsize=32768,wsize=32768,tcp,noac,forcedirectio,vers=3

4.2.4 Creating Required Directories for Oracle Clusterware Files on Shared File Systems

Use the following instructions to create directories for Oracle Clusterware files. You can also configure shared file systems for the Oracle Database and recovery files.

To create directories for the Oracle Clusterware files on separate file systems from the Oracle base directory, follow these steps:

1. If necessary, configure the shared file systems that you want to use and mount them on each node.

   Note:

   The mount point that you use for the file system must be identical on each node. Ensure that the file systems are configured to mount automatically when a node restarts.

2. Use the df -k command to determine the free disk space on each mounted file system.

3. From the display, identify the file systems that you want to use:

   File Type	File System Requirements
   Oracle Clusterware files	Choose a file system with at least 560 MB of free disk space (one OCR and one voting disk, with external redundancy).

   
   

   If you are using the same file system for more than one type of file, then add the disk space requirements for each type to determine the total disk space requirement.

4. Note the names of the mount point directories for the file systems that you identified.

5. If the user performing installation (crs or oracle) has permissions to create directories on the disks where you plan to install Oracle Clusterware and Oracle Database, then OUI creates the Oracle Clusterware file directory

   If the user performing installation does not have write access, then you must create these directories manually using commands similar to the following to create the recommended subdirectories in each of the mount point directories and set the appropriate owner, group, and permissions on them:

   • Example of creating Oracle Clusterware file directory owned by the installation user oracle:

     # mkdir /mount_point/oracrs
     # chown oracle:oinstall /mount_point/oracrs
     # chmod 775 /mount_point/oracrs
     

By making the crs or oracle user the owner of these directories, this permits them to be read by multiple Oracle homes, including those with different OSDBA groups.

When you have completed creating subdirectories in each of the mount point directories, and set the appropriate owner, group, and permissions, you have completed CFS or NFS configuration.

4.3.1 Identifying Required Raw Partitions for Clusterware Files

Table 4-2 lists the number and size of the raw partitions that you must configure for Oracle Clusterware files.