| Oracle® Exalytics In-Memory Machine Owner's Guide 11g Release 1 (11.1.1.6.0) Part Number E25369-01 |
|
|
PDF · Mobi · ePub |
| Oracle® Exalytics In-Memory Machine Owner's Guide 11g Release 1 (11.1.1.6.0) Part Number E25369-01 |
|
|
PDF · Mobi · ePub |
This chapter describes how to connect an Oracle Exalytics In-Memory Machine and an Oracle Exadata Database Machine together on the same InfiniBand fabric.
It contains the following topics:
This section contains the following topics:
Each Oracle Oracle Exalytics In-Memory Machine includes an InfiniBand card, such as the 2-port QDR InfiniBand Host Channel Adapter (HCA). The Mellanox part number for this card is MHQH29B-XTR.
This InfiniBand card is installed in the Oracle Oracle Exalytics In-Memory Machine in its PCIe Slot 6.
You may use any spare 5-meter InfiniBand cables included in your Oracle Exadata Database Machine. If you need assistance, contact Oracle support personnel. You need these cables for connecting your Oracle Oracle Exalytics In-Memory Machine to an Oracle Exadata Database Machine on the same InfiniBand fabric.
Oracle Exadata Database Machine includes two 36-port InfiniBand leaf switches (Sun Datacenter QDR InfiniBand Switch 36). These two switches located in the middle of the rack are referred to as leaf switches. A spine switch is included in unit 1 of Oracle Exadata Database Machine Full Rack and Half Rack configurations.
Note:
In this document, the leaf switch located in the lower unit of the Oracle Exadata Database Machine rack is referred to as leaf 1. The leaf switch located above leaf 1 in the Oracle Exadata Database Machine rack is referred to as leaf 2.
For more information about Oracle Exadata Database Machine's rack layout and hardware components, see the Oracle Exadata Database Machine Owner's Guide.
This section describes the following topics:
Scenario 1: Single Exalytics In-Memory Machine with Oracle Exadata Database Machine
Scenario 2: Clustered Exalytics In-Memory Machine with Oracle Exadata Database Machine
It is assumed that the rack containing the Oracle Oracle Exalytics In-Memory Machine and the Oracle Exadata Database Machine rack are placed side by side in the data center. Consider the length of the connecting InfiniBand cables when placing the racks in your data center. Do not place the Oracle Oracle Exalytics In-Memory Machine in an Oracle Exadata Database Machine rack.
Figure 6-1 shows how a single (non-clustered) Oracle Oracle Exalytics In-Memory Machine is connected to an Oracle Exadata Database Machine rack.
Figure 6-1 Non-Clustered Oracle Exalytics In-Memory Machine Connected to Oracle Exadata Database Machine
For high availability purposes, connect the two ports of the InfiniBand card, which is installed in the Oracle Oracle Exalytics In-Memory Machine, to separate leaf switches of the Oracle Exadata Database Machine. For example, as shown in Figure 6-1, you can connect port 1 of the Oracle Oracle Exalytics In-Memory Machine's InfiniBand card to the leaf 1 switch (Sun Datacenter InfiniBand Switch 36 installed in the lower unit of the Oracle Exadata Database Machine). In addition, you should connect port 2 of the Oracle Oracle Exalytics In-Memory Machine's InfiniBand card to the leaf 2 switch (Sun Datacenter InfiniBand Switch 36 installed in the upper unit of the Oracle Exadata Database Machine). Port 1 can function as the active port, and port 2 can function as the passive port.
Note:
On the leaft switches of the Oracle Exadata Database Machine, the following free ports can be used:
Ports 5B, 6A, 6B, 7A, 7B, and 12A
Figure 6-2 shows how a cluster of Oracle Oracle Exalytics In-Memory Machines is connected to an Oracle Exadata Database Machine rack.
Figure 6-2 Clustered Oracle Exalytics In-Memory Machine Connected to Oracle Exadata Database Machine
For high availability purposes, connect the active port of the InfiniBand card included in each of the Oracle Oracle Exalytics In-Memory Machines to the leaf 1 switch (Sun Datacenter InfiniBand Switch 36 installed in the lower unit of the Oracle Exadata Database Machine). In addition, you should connect the passive port of the InfiniBand card included in each of the Oracle Oracle Exalytics In-Memory Machines to the leaf 2 switch (Sun Datacenter InfiniBand Switch 36 installed in the upper unit of the Oracle Exadata Database Machine). Port 1 can function as the active port, and port 2 can function as the passive port.
Note:
On the leaft switches of the Oracle Exadata Database Machine, the following free ports can be used:
Ports 5B, 6A,6B,7A,7B, and 12A
Each InfiniBand network requires a Subnet Manager (SM). You can use the SM running on one of the Oracle Exadata Database Machine's leaf switches in your fabric.
For more information about using the InfiniBand switch, see the following documents:
The kernel modules required for InfiniBand network support are loaded by the Oracle Linux base operating system, which is pre-installed on the Oracle Oracle Exalytics In-Memory Machine.
You can monitor many aspects of the InfiniBand fabric by running InfiniBand commands on the Sun Datacenter InfiniBand Switch 36, which is included in the Oracle Exadata Database Machine.
For more information about these commands, see the Sun Datacenter InfiniBand Switch 36 Command Reference.
After establishing InfiniBand network connectivity, you can verify the status of the InfiniBand link by using the ibstat command on the CLI of the switch.
The ibstat command displays the basic status, as in the following example:
# ibstat Switch 'is4_0' Switch type: MT48436 Number of ports: 0 Firmware version: 7.3.0 Hardware version: a0 Node GUID: 0x00212856cfe2c0a0 System image GUID: 0x00212856cfe2c0a3 Port 0: State: Active Physical state: LinkUp Rate: 40 Base lid: 16 LMC: 0 SM lid: 16 Capability mask: 0x4250084a Port GUID: 0x00212856cfe2c0a0
The physical state of connected ports is shown as LinkUp.
After verifying the basic InfiniBand network connectivity, you can run the ibhosts, ibswitches, and ibnetdiscover commands on the switch CLI.
The ibhosts command displays host nodes in the InfiniBand fabric, the ibswitches command displays the InfiniBand switch nodes in the fabric, and the ibnetdiscover command discovers the InfiniBand topology.
For more information such monitoring and diagnostic commands, see the Sun Datacenter InfiniBand Switch 36 Command Reference.
|
 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Legal Notices |
|
