1 Deployment Overview

1.1.1 Supported Component Versions

The following component versions are supported for Enterprise Manager Cloud Control 12c:

• Exadata Storage Server Software 11g Release 2 (11.2.2.3.0 through 11.2.3.2)

• InfiniBand Switch Release 1.1.3.0.0 to 1.3.3.2.0.

• Integrated Lights Out Manager (ILOM) Release 3.0.9.27.a r58740

  • ILOM ipmitool Release 1.8.10.3 (for Oracle Linux) or Release 1.8.10.4 (for Oracle Solaris)

• Avocent MergePoint Unity KVM Switch Release 1.2.8

• Power Distribution Unit Release 1.04

• Cisco - Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500-IPBASE-M), Version 12.2(46)SG, RELEASE SOFTWARE (fc1)

1.1.2 Supported Operating Systems

The following operating systems (where OMS and agent is installed on) are supported:

• Management Server Plug-in: (all Enterprise Manager Cloud Control 12c-certified platforms) specifically, as of now Linux x86, Linux x86-64, Oracle Solaris on SPARC (64-bit), Oracle Solaris on x86-64 (64-bit)

• Agent Plug-in: Linux x86-64, Oracle Solaris on x86-64 (64-bit)

1.1.3 Oracle Exadata Database Machine Hardware Not Supported

The following Oracle Exadata Database Machine hardware configurations are not supported for Enterprise Manager Cloud Control 12c or 12c Bundle Pack 1 (BP1):

• V1 hardware

• Super Clusters

• Expansion rack

• Multi-Rack Exadata Database Machine

  Note:

  Although Multi-Rack Exadata Database Machine display in schematic is not supported or monitored as a single entity. However, each individual Rack can be monitored and managed separately.

• Use of client network hostnames for compute nodes as Enterprise Manager target names is not supported with the Exadata plug-in Release 12.1.0.1 and 12.1.0.2. This support is added in 12.1.0.3.

1.2.1 Create a Database Server ILOM Service Processor User

For the Enterprise Manager agent to communicate with an ILOM service processor, there must be a specific user ID established on the ILOM service processor.

The specific ILOM user ID can be added in the ILOM service processor web interface, ILOM CLI, or with the ipmitool command. This example uses ILOM CLI.

For security reasons, the password to the ILOM service processor root user ID does not appear in the ILOM CLI commands in this example. Create a text file that contains one line with the password for the ILOM service processor root user ID in the /tmp directory called changeme. The /tmp/changeme file is used in the ILOM CLI command examples.

1. Log in to the Service Processor as root:

   ssh root@[Service Processor IP]
   Password:
   

2. Change to the users directory:

   cd /SP/users
   

3. Create the oemuser user and password:

   create oemuser
   
   Creating user...
   Enter new password: ********
   Enter new password again: ********
   
   Created /SP/users/oemuser
   

4. Change to the new user's directory and set the role:

   cd oemuser
   /SP/users/oemuser
   
   set role='aucro'
   Set 'role' to 'aucro'
   

5. Test the ILOM user ID created in step 3 by listing the last 10 system events:

   ipmitool -I lan -H <ilom_hostname> -U oemuser -P oempasswd -L USER sel list last 10
   

6. Repeat steps 1 through 5 for the rest of the compute node ILOM service processors in your Oracle Database Machine.

1.2.2 Verify Software Versions

Exadata Storage Server Software

The supported version is Exadata Storage Server Software 11g Release 2 (11.2.2.3.0 through 11.2.3.2). To verify the cell software version on the Exadata cell, ssh to the Exadata cell as the root, celladmin, or cellmonitor user. Run:

cellcli -e 'list cell detail'

Look for "releaseVersion" in the output.

ILOM ipmitool

Enterprise Manager requires a minimal ipmitool software version of 1.8.10.4 or later for Oracle Solaris and 1.8.10.3 for Oracle Linux. To view the software version:

• For Oracle Linux, run the following command as the root user on one of the database servers in the cluster:

  dcli -g ~/dbs_group -l root ipmitool –V
  

  Note:

  The dbs_group file contains the list of compute node hostnames, one on each line. If the file does not exist, you can create it before running the dcli command.

• For Oracle Solaris, run the following command on each of the database servers in the cluster:

  /opt/ipmitool/bin/ipmitool -V
  

InfiniBand Switch

To verify the version of the InfiniBand switch firmware in your environment:

1. Log on to the management interface for the InfiniBand Switch (using ssh).

2. Run the following command:

   nm2version
   

   The output should be similar to this:

   # nm2version
   Sun DCS 36p version: 1.1.3-2
   

   This example shows a supported configuration for deploying the plug-in to monitor.

3. If the nm2version command returns output similar to this:

   # nm2version
   NM2-36p version: 1.0.1-1
   

   Then you must upgrade your InfiniBand switch firmware. Follow the instructions listed in My Oracle Support (MOS) Document 888828.1:

   https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=REFERENCE&id=88882
   

PDU Firmware

The PDU firmware version must be 1.02 or later. The current version can be obtained by logging into the web interface of the PDU. On the left side of the screen, click Module Info to view the PDU firmware version.

Software updates for the PDU are available at:

https://updates.oracle.com/Orion/PatchDetails/process_form?patch_num=12871297

KVM Application

The KVM Application software version must be 1.2.8 or later. The current version can be obtained by logging into the web interface of the KVM. On the left side of the screen under Unit View, Appliance, Appliance Settings, click Versions to view the Application software version.

Software updates for KVM are available at:

http://www.avocent.com/Support_Firmware/MergePoint_Unity/MergePoint_Unity_Switch.aspx

1.2.3 Verify Names Resolution

The Enterprise Manager OMS server(s) require direct network access to each of the compute nodes. If the names of the compute nodes are not registered in the OMS nodes' DNS, then they will have to be manually entered in the /etc/hosts file for each OMS.

Each compute node should be verified to be able to resolve the hostnames of the ILOM servers, PDU's, storage cell nodes, and InfiniBand and Cisco switches. Again, if the names of those components are not registered in DNS, then entries can be added to the /etc/hosts file of each compute node.

To manage the Exadata Database Machine components from Enterprise Manager Cloud Control 12c, it is necessary for your local machine to be able to resolve the host name of Cloud Control 12c.

To access any of the Exadata Database Machine components directly from your local machine, it is also necessary for your local machine to be able to resolve the names of those components.

1.2.4 Verify Firewall Configuration

To verify the firewall configuration:

1. Allow ping

   In many secure network environments, it is normal for the ping service to be disabled. Enterprise Manager uses ping to establish the basic availability and status of the Exadata Database Machine components.

   • The compute nodes need to have the ping service and port enabled from the OMS Server(s).

   • All other Exadata Database Machine components (ILOM servers, PDU's, storage cell nodes, InfiniBand switches, and Cisco switch) need to have the ping service and port enabled from the compute nodes (where the agents are running).

   Note:

   The ping traffic overhead is minimal. The agent pings the targets every five minutes.

2. Open Database Ports

   The database listener ports must be opened for the Enterprise Manager OMS server(s). Note that Exadata Database Machine databases will use SCAN listeners; so, ports will need to be opened for the base compute node, the compute node virtual IP, and scan listeners addresses.

   For example, if an Exadata Database Machine quarter rack has been configured with two compute nodes - exadbnode1.example.com and exadbnode2.example.com - and the listeners are using port 1521, then port 1521 will have to be opened to the Enterprise Manager Server for the following addresses:

   • The compute node hostnames - exadbnode1.example.com and exadbnode2.example.com

   • The virtual IPs for each compute node - exadbnode1-vip.example.com and exadbnode1-vip.example.com

   • The scan listener hostname - scan-exadatadb

3. Open Enterprise Manager Upload Port

   The Enterprise Manager Cloud Control 12c agents require access to the Enterprise Manager Servers upload service, normally configured on port 4889 for HTTP uploads and 4900 for HTTPS. To verify the ports assigned, run the following command on the OMS server command line.

   emctl status oms -details
   

   These ports will need to be opened for each of the compute nodes.

4. Open Agent Ports

   The OMS server(s) will need to be able to connect to the Enterprise Manager Cloud Control 12c Agent HTTP/HTTPS port on each compute node. The Agent port defaults to 3872. If port 3872 is not available, the next available port starting from port 1830 is used.

   To identify the port used:

   • Run the following command on the compute node command line:

     emctl status agent
     

   • Alternatively, you can look for the value of the EMD_URL property in the emd.properties file under <AGENT_HOME>/agent_inst/sysman/config

5. Open SSH Ports

   The Enterprise Manager Cloud Control 12c Agents require ssh access to the Exadata Database Machine components they monitor. As the Agents will run on the compute nodes the ssh ports, 22, on each of the storage cells, ILOMs, PDUs, KVMs, InfiniBand switches, and Cisco switch will need to be opened for each of the compute nodes.

   Note:

   The emkit configures ssh access but still requires the ports to be opened first.

6. Allow UDP Traffic (SNMP Ports)

   All Exadata Database Machine components need to be able to send SNMP traps to the Agents running on the compute nodes. SNMP uses the UDP protocol so the Agent port and port 162 need to be opened for UDP traffic between the Storage Cells, ILOMs, InfiniBand Switches, Cisco Switch, KVM, and PDUs and the Agent.

1.4.1 Agent Installation Prerequisite - Solaris 11 Only

If you are running Exadata Storage Server Software Release 11g Release 2 (11.2.3.1) on Oracle Solaris, follow the steps below before installing the agent.

1. Log in to the compute node host as agent user and execute the ppriv $$ command. This command displays the E (Effective), P (Permitted), I (Inherited), and L (Limited) privileges.

   Generally, E, P, and I have the same set of basic privileges and for any normal user.

2. Assuming that output of the above command shows basic privileges, add the priv_sys_net_config privilege to the agent user so that InfiniBand commands can be executed.

   Log in as root and run:

   usermod -K defaultpriv=basic,priv_sys_net_config <agent_user>
   

   This command adds sys_net_config to the agent user.

3. Log in as the agent user again and run the ppriv $$ command to verify if the privileges are set. The output will show the sys_net_config privilege along with the existing (basic) set of privileges.

1.4.2 Install Oracle Database and Oracle Exadata Plug-ins Using the Automated Kit

The Oracle Enterprise Manager Cloud Control 12c Setup Automation kit is available for download from My Oracle Support as a patch. The kit simplifies the process of deploying the agents on each of the compute nodes considerably, allowing agents to be deployed to all of the compute nodes, in one go, from one of the compute nodes, using one simple command. Instructions for using the kit to deploy agents on the compute nodes are provided in the patch README.txt.

To download the Automation kit, see Doc ID 1440951.1 in My Oracle Support:

https://support.oracle.com

1.4.3 Install Enterprise Manager Agent Using the Push Method

The installation instructions were captured on a Quarter Rack configuration (for example, 2 compute nodes, 3 cells).

1. Add the Exadata Database Machine compute nodes as host targets to Oracle Enterprise Manager Cloud Control 12c. From the Enterprise Manager home page, click the Setup menu (upper right corner), Add Target, then Add Targets Manually.

2. On the Add Host Targets: Host and Platform screen, specify a session name. Then identify the fully qualified hostnames and select the platform. Figure 1-1 shows how the selection should look.

   Figure 1-1 Add Host Targets

   
   

3. Click Next to add the details for the host.

4. On the Installation Details screen, provide the following information:

   • Installation Base Directory

   • Instance Directory

   • Named Credential

   • For Port, leave this field blank. As part of the installation process, an available port will be selected automatically.

5. Click Next to review the details about the host.

6. Click Deploy Agent to start the agent deployment process.

7. As the deployment process continues, remote prerequisite checks are automatically checked. If there are no issues, you will be presented with an Agent Deployment Summary with an indication that the agent deployment has passed. Figure 1-2 shows an example of a successful agent deployment.

   Figure 1-2 Agent Deployment Summary

   
   

http://docs.oracle.com/cd/E24628_01/install.121/e24089/install_agent_usng_rsp.htm#CEGGACJE

1.4.4 Manually Deploy Exadata Plug-in

Under the following conditions, you may need to manually deploy the Exadata plug-in on each of the compute nodes:

• For a new or fresh installation: Deploy the plug-in manually if you did not deploy the agent using Automation kit or push the latest version of the agent through OMS.

• For an upgrade to an existing installation: Deploy the Exadata plug-in manually if an older version of the Exadata plug-in has been deployed to the agent already and you would like to upgrade to the latest version of the Exadata plug-in.

To determine if the Exadata plug-in is deployed on each compute node and what version it is, you have two options:

• From a terminal window, run the following command:

  emctl listplugins agent
  

  Note:

  The emctl listplugins agent command must be run on the compute node using the emctl in the agent installation directory.

• From Enterprise Manager Cloud Control, click the Setup menu (upper right corner), Extensibility, and then Plug-ins.

To manually deploy the Exadata plug-in:

1. From the Enterprise Manager home page, click the Setup menu (upper right corner), Extensibility, and then Plug-ins.

2. On the Plug-ins page, select Oracle Exadata from the Name list.

   Note:

   The selected version should be for Release 12.1.0.1, 12.1.0.2, or 12.1.0.3. Oracle recommends that you deploy the latest version of the Exadata and Database plug-ins to the agent.

3. With Oracle Exadata selected, click Deploy On, then Management Agent.

4. On the Deploy Plug-in on Management Agent pop-up, click Add. A search pop-up will appear to allow you to search for targets to add. In the Target Type drop-down, select Agent, then click Search.

   Select a Target name from the results list and click Select. Repeat for each agent target.

5. After you have added the agents, click Next on the Deploy Plug-in on Management Agent screen to review and verify the agent information.

6. Click Deploy to deploy the plug-in to the agents.

7. Once the agent has deployed to all the agents, a confirmation screen will appear. Click OK to dismiss the pop-up or Show Status to display the status of the agent in the Enterprise Manager Deployment Activities screen.